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OBJECTIVES 


We  defined  and  we  pursued  a  novel  line  of  research  that  lies  in  a  multidis¬ 
ciplinary  intersection  of  Estimation  Theory,  Communications  Theory,  and 
Mean-Square  optimum  linear  filtering.  Consider  an  arbitrary  input  signal 
vector  space  and  a  given  information  bearing  signal  vector  to  be  protected 
or  recovered  in  the  presence  of  multiuser  or  other  forms  of  heavy  interfer¬ 
ence.  Based  strictly  on  statistical  conditional  optimization  principles,  we 
developed  an  iterative  algorithm  that  starts  from  the  conventional  matched- 
filter  correlator  and  generates  a  sequence  of  linear  filters  ( “auxiliary- vector” 
filters)  that  converges  to  the  exact  MS-optimum  solution.  At  each  itera¬ 
tion  step,  the  filter  is  given  as  a  direct  function  of  the  input  autocorrelation 
matrix,  the  signal  vector  waveform  to  be  protected,  and  the  filter  at  the 
previous  iteration.  When  the  autocorrelation  matrix  is  sample-average  esti¬ 
mated  from  a  short  data  record,  this  procedure  offers  the  means  for  effective 
control  over  the  filter  estimator  bias  versus  (co-)variance  trade-off.  For  a 
fixed  data  record  size  the  filter  estimators  in  this  sequence  have  rapidly  de¬ 
creasing  bias  and  slowly  increasing  variance.  They  outperform  other  known 
estimators  such  as  Sample-Matrix-Inversion  (SMI),  Diagonal-Loading  (DL) 
SMI,  RLS,  LMS,  reduced-rank  eigenvector  decomposition  and  “multistage” 
nested  Wiener  filter.  While  all  of  the  above  estimators  converge  to  the  op¬ 
timum  MMSE/MVDR  solution  for  infinitely  long  data  record,  for  any  given 
finite  data  set  there  is  at  least  one  AV  filter  estimator  in  the  sequence  that 
outperforms  the  SMI,  DL-SMI,  RLS,  LMS,  reduced-rank  eigenvector  decom¬ 
position  as  well  as  the  multistage  nested  Wiener  filter. 

The  theoretical  and  practical  implications  of  these  results  are  far  reaching. 
Biased  estimators  and  algorithms  that  offer  full  control  over  the  bias/ variance 
balance  are  rarely  reported  in  the  literature,  if  any  in  a  communications  appli¬ 
cable  context.  We  pursued  thorough  theoretical  analysis  of  the  filter  estima¬ 
tor  sequence  and  application  for  rapid  adaptive  interference  suppression  to: 
spread-spectrum/CDMA  communications  systems,  spread-spectrum/ CDMA 
systems  with  adaptive  antenna  arrays,  interference  resistant  synchronization, 
and  jam-resistant  GPS. 

This  work  was  taken  one  step  further  and  the  short-data-record  adaptive 
optimization  problem  was  studied  in  the  context  of  non-linear  processing  with 
applications  to  multi-layer  perceptron  receiver  designs  for  spread-spectrum 
communications.  The  importance  of  the  theoretical  developments  lies  in  the 
fact  that  they  are  independent  of  the  specific  receiver  optimizations  criterion 
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and  thus  many  existing  receiver  optimization  algorithms  can  benefit  by  these 
developments  directly. 

Finally,  we  looked  at  the  problem  of  binary  code  design.  Although  this 
investigation  is  at  an  initial  stage,  our  theoretical  developments  so  far  ad¬ 
vanced  the  prior-state-of-the-art  in  the  context  of  fundamental  binary  code 
design  for  code  division  signal  multiplexing.  We  developed  novel,  doubly 
optimal  designs  of  minimum  total-squared-correlation  (TSC)  and  minimum 
maximum-squared-correlation  (MSC)  binary  codes  and  we  pursued  theoret¬ 
ical  analysis  of  the  codes  and  application  for  interference  avoidance  at  the 
transmitter-end  of  a  communication  link. 


2  SUMMARY  OF  RESEARCH  EFFORT 

The  milestones  reached  during  the  duration  of  this  research  project  (2  years) 
are  as  follows. 

(i)  We  addressed  successfully  the  problem  of  determining  an  automated  the¬ 
oretically  derived  rule  for  the  selection  of  the  best  number  of  auxiliary  vectors 
for  a  given  data  record  of  input  observations. 

(ii)  We  developed  a  recursive  algorithm  for  the  on-line  estimation  of  the  AV 
filter  and  a  modified  RLS-type  algorithm  for  the  estimation  of  the  MMSE/ 
MVDR  filter  that  are  both  based  on  the  interference-plus-noise  covariance 
matrix  estimate  and  we  established  formally  their  convergence  properties. 
(in)  We  investigated  theoretically  the  coarse  synchronization  performance 
of  blind  adaptive  linear  self-synchronized  receivers  for  asynchronous  direct- 
sequence  spread-spectrum  communications  under  finite  data  record  adapta¬ 
tion.  We  derived  analytical  expressions  that  approximate  the  probability 
of  coarse  synchronization  error  of  matched-filter-type  (MF)  and  minimum- 
variance-distortionless-response-type  (MVDR)  receivers  based  on  transfor¬ 
mation  noise  modeling  techniques. 

(iv)  We  studied  theoretically  the  data-record-size  requirements  of  MMSE/ 
MVDR-type  adaptive  algorithms  to  meet  a  given  performance  objective  in 
joint  space-time  signal  detection  and  direction-of-arrival  (DoA)  estimation 
problems  for  direct-sequence  spread-spectrum  systems.  We  derived  closed 
form  expressions  that  provide  the  data  record  size  that  is  necessary  to  achieve 
a  given  performance  confidence  level  in  a  neighborhood  of  the  optimal  per¬ 
formance  point,  as  well  as  expressions  that  identify  the  performance  level 
that  can  be  reached  for  a  given  data  record  size. 
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(v)  As  an  application  of  great  importance  to  AFRL,  we  addressed  the  prob¬ 
lem  of  navigation  data  demodulation  by  an  adaptive  GPS  receiver  that  uti¬ 
lizes  a  bank  of  single-satellite  linear-tap-delay  filters  and  employs  antenna- 
array  reception. 

(vi)  We  investigated  the  relative  output  SINR  performance  of  the  full  decor¬ 
relator  and  the  partial  decorrelator  spread-spectrum  receiver.  For  each  re¬ 
ceiver,  we  considered  two  implementations  that  are  equivalent  under  perfectly 
known  input  statistics.  The  first  implementation  utilizes  the  signature  ma¬ 
trix  while  the  second  implementation  is  based  on  the  eigen-decomposition 
of  the  ideal  input  covariance  matrix.  While  the  full  decorrelator  aims  at 
decorrelating  the  complete  spread-spectrum  (multiple  access)  interference, 
the  partial  decorrelator  aims  at  decorrelating  only  a  part  of  it  by  excluding 
one  or  more  user-signatures  or  eigenvectors  from  the  corresponding  imple¬ 
mentation  method.  We  derived  necessary  and  sufficient  conditions  on  the 
signal  energy  and  signature  cross-correlation  levels  under  which  the  partial 
decorrelator  outperforms  the  full  decorrelator  in  the  output  SINR  sense.  Nu¬ 
merical  results  demonstrated  the  validity  of  the  above  conditions  and  simu¬ 
lation  studies  illustrated  the  relative  SINR  and  BER  performance  of  the  full 
and  partial  decorrelators  under  sample-average-estimated  input  statistics. 

(vii)  The  short-data-record  adaptive  filtering  problem  was  taken  one  step 
further  and  studied  in  the  context  of  non-liner  signal  processing.  We  ex¬ 
amined  multi-layer  perceptron  DS-CDMA  receiver  designs  and  we  developed 
a  fast  converging  training  methodology  that  is  independent  of  the  specific 
optimization  criterion  used  to  train  the  adaptive  receiver. 

(viii)  In  direct-sequence  spread-spectrum  systems,  the  pre-detection  signal- 
to-interference  -plus-noise  ratio  (SINR)  at  the  output  of  the  single-user  minimum- 
mean-square-error  (MMSE)  or  auxiliary- vector  (AV)  filter  is  a  function  of  the 
specific  user  spreading  code  (signature).  We  considered  the  adaptive  opti¬ 
mization  of  the  user  signature  assignment  such  that  the  output  SINR  of  the 
MMSE/AV  filter  is  maximized  under  a  transmitter  power  constraint.  In  the 
context  of  binary  signatures,  the  complexity  of  the  signature  optimization 
procedure  is  exponential  in  the  processing  gain.  We  derived  a  low-cost  subop¬ 
timum  adaptive  binary  signature  assignment  algorithm  based  on  conditional 
optimization  principles.  We  used  this  algorithm  to  design  an  efficient  system- 
wide  multiuser  adaptive  signature  set  assignment  scheme.  The  performance 
of  the  proposed  scheme  was  evaluated  under  asynchronous  multipath  fading 
DS-CDMA  channel  models  and  was  compared  to  the  performance  of  systems 
with  arbitrarily  chosen  signature  sets. 
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(ix)  Transmitter  optimization  was  further  investigated  in  the  context  of  op¬ 
timal  spreading  code  design.  We  designed  doubly  optimal  binary  code  sets, 
that  is,  sets  of  binary  spreading  sequences  that  can  be  used  for  code  divi¬ 
sion  multiplexing  purposes  that  exhibit  minimum  total-squared-correlation 
(TSC)  and  minimum  maximum-squared-correlation  (MSC)  at  the  same  time. 

(x)  We  showed  that  we  can  efficiently  and  effectively  approach  the  error  rate 
performance  of  the  optimum  multiuser  detector  as  follows.  We  utilized  a 
multiuser  decorrelating  or  MMSE/AV  filter  as  a  pre-processor  and  we  estab¬ 
lished  that  the  output  magnitudes,  when  properly  scaled,  provide  a  reliability 
measure  for  each  user  bit  decision.  An  ordered  reliability-based  error  search 
sequence  of  length  linear  in  the  number  of  users  returns  the  most  likely  user 
bit  vector  among  all  visited  options.  Numerical  and  simulation  studies  for 
moderately  loaded  systems  that  permit  exact  implementation  of  the  optimum 
detector  indicated  that  the  error  rate  performance  of  the  optimum  and  the 
proposed  detector  are  nearly  indistinguishable  over  the  whole  pre-detection 
signal-to-noise  ratio  (SNR)  range  of  practical  interest.  Similar  studies  for 
higher  user  loads  (that  prohibit  comparisons  with  the  optimum  detector) 
demonstrated  error  rate  performance  gains  of  orders  of  magnitude  in  com¬ 
parison  with  straight  decorrelating  or  MMSE  multiuser  detection. 


3  ACCOMPLISHMENTS/NEW  FINDINGS 

Selecting  the  most  successful  (in  some  appropriate  sense)  AV  filter  estima¬ 
tor  in  the  generated  sequence  for  a  given  data  record  was  a  problem  that 
had  not  been  addressed  so  far.  We  dealt  exactly  with  this  problem  and  we 
proposed  two  data  driven  selection  criteria  [1] ,  [2] ,  [3] .  The  first  criterion  is 
specific  to  digital  communications  (binary  hypothesis  testing)  and  maximizes 
the  estimated  J-divergence  of  the  AV-filter  output  conditional  distributions 
given  the  transmitted  information  bit.  The  second  criterion  applies  to  all 
filter  estimation  problems  and  minimizes  the  cross-validated  sample  average 
variance  of  the  AV-filter  output. 

We  observed  that  the  presence  of  the  desired  signal  during  estimation  of 
the  minimum-mean-square-error  (MMSE) /minimum- variance-distortionless- 
response  (MVDR)  and  auxiliary-vector  (AV)  filters  under  limited  data  sup¬ 
port  leads  to  significant  signal-to-interference-plus-noise  ratio  (SINR)  per¬ 
formance  degradation.  We  quantified  this  observation  in  the  context  of 
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DS/CDMA  communications  by  deriving  close  approximations  for  the  mean- 
square  filter  estimation  error,  the  probability  density  function  of  the  output 
SINR  and  the  probability  density  function  of  the  symbol-error-rate  (SER) 
of  the  sample-matrix-inversion  (SMI)  receiver  evaluated  using  both  desired- 
signal- “present”  and  desired-signal- “absent”  input  covariance  matrix.  To 
avoid  such  performance  degradation  we  proposed  a  DS/CDMA  receiver  that 
utilizes  a  simple  pilot-assisted  algorithm  that  estimates  and  then  subtracts 
the  desired  signal  component  from  the  received  signal  prior  to  filter  estima¬ 
tion.  Then,  to  accommodate  decision  directed  operation  we  developed  two 
recursive  algorithms  for  the  on-line  estimation  of  the  AV  and  MMSE/MVDR 
filter  and  we  studied  their  convergence  properties  [4]. 

Equipped  with  these  important  developments,  we  considered  blind  adap¬ 
tive  linear  receivers  for  the  demodulation  of  DS/SS  signals  in  asynchronous 
transmissions.  The  proposed  structures  are  self-synchronized  in  the  sense 
that  adaptive  synchronization  and  demodulation  are  viewed  and  treated  as 
an  integrated  receiver  operation.  Two  computationally  efficient  combined 
synchronization/demodulation  schemes  were  proposed,  developed  and  ana¬ 
lyzed  [5].  The  first  scheme  is  based  on  the  principles  of  minimum- variance- 
distortionless-response  (MVDR)  processing,  while  the  second  scheme  follows 
the  principles  of  auxiliary- vector  filtering  and  exhibits  enhanced  performance 
in  short  data  record  scenarios.  In  both  cases  the  resulting  receiver  is  a  linear 
structure  of  order  exactly  equal  to  the  system  processing  gain.  Simulation 
studies  demonstrated  the  coarse  synchronization,  as  well  as  the  bit-error- 
rate  performance  of  the  proposed  strategies.  In  addition,  we  investigated 
theoretically  the  coarse  synchronization  performance  of  blind  adaptive  linear 
self-synchronized  receivers  for  asynchronous  direct-sequence  code-division- 
multiple-access  communications  under  finite  data  record  adaptation.  We 
derived  [6]  analytical  expressions  that  approximate  the  probability  of  coarse 
synchronization  error  of  matched-filter-type  (MF)  and  minimum- variance- 
distortionless-response- type  (MVDR)  receivers  based  on  transformation  noise 
modeling  techniques.  The  expressions  are  explicit  functions  of  the  data  record 
size  N  and  the  filter  order  p  and  reveal  the  effect  of  short-data-record  Sample- 
Matrix-Inversion  (SMI)  implementations  on  the  coarse  synchronization  per¬ 
formance.  Besides  their  theoretical  value,  the  derived  expressions  provide 
simple,  highly-accurate  alternatives  to  computationally  demanding  perfor¬ 
mance  evaluation  through  simulations.  Numerical  and  simulation  studies 
examined  the  accuracy  of  the  theoretical  developments  and  showed  that  the 
derived  expressions  approximate  closely  the  actual  coarse  synchronization 
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performance. 

We  also  investigated  the  data-record-size  requirements  of  minimum- variance- 
distortionless-response-type  adaptive  algorithms  to  meet  a  given  performance 
objective  in  joint  space-time  signal  detection  and  direction-of-arrival  (DoA) 
estimation  problems  for  direct-sequence  spread-spectrum  systems.  We  de¬ 
rived  closed  form  expressions  that  provide  the  data  record  size  that  is  nec¬ 
essary  to  achieve  a  given  performance  confidence  level  in  a  neighborhood  of 
the  optimal  performance  point,  as  well  as  expressions  that  identify  the  per¬ 
formance  level  that  can  be  reached  for  a  given  data  record  size  [7].  This  was 
done  by  utilizing  close  approximations  of  the  involved  probability  density 
functions.  The  practical  significance  of  the  derived  expressions  lies  in  the 
fact  that  the  expressions  are  functions  of  the  number  of  antenna  elements 
and  the  system  spreading  gain  only,  while  they  are  independent  of  the  ideal 
input  covariance  matrix  which  is  not  known  in  most  realistic  applications. 

An  important  application  for  AFRL  is  the  problem  of  navigation  data  de¬ 
modulation  by  an  adaptive  GPS  receiver  that  utilizes  a  bank  of  single-satellite 
linear-tap-delay  filters  and  employs  antenna-array  reception.  The  presence  of 
an  antenna  array  allows  the  receiver  to  operate  in  the  spatial  domain  in  addi¬ 
tion  to  the  temporal  (code)  domain.  We  investigated  disjoint-domain  as  well 
as  joint-domain  space-time  GPS  signal  processing  techniques  and  we  con¬ 
sidered  design  criteria  of  conventional  matched-filter  (MF)  type,  minimum- 
variance-distortionless-response  (MVDR)  type  and  auxiliary- vector  (AV)  type 
[8].  The  proposed  structures  utilize  filters  that  operate  at  a  fraction  of  the 
navigation  data  bit  period  (1  msec)  and  are  followed  by  hard-decision  detec¬ 
tors.  Hard  decisions  taken  over  a  navigation  data  bit  period  are  then  com¬ 
bined  according  to  a  simple  combining  rule  for  further  bit-error-rate  (BER) 
performance  improvements.  Analytic,  numerical  and  simulation  comparisons 
illustrated  the  relative  merits  of  the  investigated  design  alternatives. 

In  the  presence  of  additive  white  Gaussian  noise  (AWGN),  complete  elim¬ 
ination  of  the  spread-spectrum  interference  introduces  enhanced  noise  vari¬ 
ance  at  the  decorrelator’s  output.1  It  was  observed,  however,  that  at  times 
a  partial  decorrelator  (PDEC)  can  strike  an  improved  balance  between  in¬ 
terference  suppression  and  noise  enhancement  when  compared  to  the  full 
decorrelator  (DEC)  while  maintaining  a  decorrelating  structure.  That  is, 

1  Noise  variance  enhancement  at  the  decorrelator’s  output  refers  to  the  filtered  noise 
variance  experienced  at  the  output  of  a  decorrelator  that  is  distortionless  in  the  direction 
of  a  given  user-signature  of  interest  as  it  compares  to  the  noise  variance  at  the  output  of 
the  corresponding  MF. 
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instead  of  decorrelating  the  complete  multiple-access-interference  (MAI)  in 
the  system,  the  PDEC  chooses  purposefully  to  decorrelate  only  a  part  of  the 
MAI  so  that  it  exhibits  a  higher  output  signal-to-interference-plus-noise-ratio 
(SINR)  than  the  full  DEC.  We  understand  that  when  unknown  interference 
is  present  in  the  system,  decorrelating  receivers  may  operate  unintentionally 
under  partial  decorrelating  conditions.  We  identified  necessary  and  sufficient 
conditions  on  the  signal  energy  and  signature  cross-correlation  levels  under 
which  the  PDEC  outperforms  the  full  DEC  in  the  output  SINR  sense,  and  we 
showed  why  decorrelation  of  interferers  that  satisfy  such  conditions  should 
be  purposefully  avoided  if  at  all  possible  [9],  [10]. 

In  the  context  of  short-data-record  adaptive  non-linear  signal  processing 
of  direct-sequence  spread-spectrum  (DS-SS)  signals,  we  proved  formally  that 
the  optimum  (nonlinear)  DS-CDMA  single-user  decision  boundary  exhibits 
the  following  properties  [11]:  (a)  It  is  symmetric  with  respect  to  the  origin 
and  (b)  as  it  is  traversed  away  from  the  origin  it  converges  to  a  hyperplane 
parallel  to  the  matched  filter  (MF)  decision  boundary.  Then,  we  translated 
these  properties  to  a  set  of  constraints  that  can  be  used  by  any  optimization 
algorithm  for  the  selection  (training)  of  the  parameters  of  a  general  multilayer 
perceptron  neural  network  receiver.  Using  these  constraints,  the  number  of 
parameters  to  be  optimized  is  reduced  by  nearly  50%  for  large-size  networks 
which  effectively  doubles  the  speed  of  the  training  procedure.  Furthermore, 
we  utilized  the  above  properties  to  develop  a  new  initialization  scheme  that 
provides  additional  improvements  on  the  convergence  rate  and  can  be  used 
by  any  recursive  optimization  algorithm.  As  a  representative  case  study, 
we  considered  the  familiar  backpropagation  (BP)  algorithm  and  developed  a 
new  version  of  BP  that  incorporates  both  the  proposed  constraints  and  the 
proposed  initialization.  Convergence  rate  enhancement  of  about  two  orders 
of  magnitude  was  achieved.  We  also  developed  a  fast  converging  adaptive 
training  algorithm  that  minimizes  the  bit  error  rate  (BER)  at  the  output 
of  the  receiver  [12],  [13],  [14].  It  is  important  to  note  that,  in  addition  to 
utilizing  the  constraints  that  were  derived  from  the  properties  of  the  opti¬ 
mum  single-user  decision  boundary  the  minimum  BER  adaptive  algorithm 
(a)  incorporates  the  BER,  i.e.  the  ultimate  performance  evaluation  mea¬ 
sure,  directly  into  the  learning  process,  and  (b)  embeds  importance  sampling 
principles  directly  into  the  receiver  optimization  process. 

We  considered  the  problem  of  binary  signature  set  design  for  DS-CDMA 
users  that  transmit  over  multipath  fading  channels.  First,  we  considered  the 
problem  of  selecting  optimally  the  signature  of  a  single  user  in  the  presence 
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of  spread-spectrum  interference  (MAI),  inter-symbol-interference  (ISI)  due 
to  the  multipath  channel,  and  background  noise.  The  signature  is  chosen  to 
maximize  the  output  SINR  of  the  ( “maximum-SINR” )  RAKE/MMSE  filter. 
Since  optimum  binary  signature  selection  has  complexity  exponential  in  the 
processing  gain  of  the  system,  we  proposed  [15]  a  suboptimum  computation¬ 
ally  efficient  algorithm  based  on  the  Cholesky  decomposition  method.  The 
proposed  algorithm  optimizes  the  coordinates  of  the  user  signature  condition¬ 
ally  and  is  characterized  by  linear  complexity.  Finally,  we  generalized  this 
approach  to  cover  the  system-wide  multiuser  signature  set  selection  problem, 
where  each  user’s  signature  is  adjusted  iteratively  according  to  the  above  al¬ 
gorithm.  The  performance  of  this  scheme  was  evaluated  through  simulations 
where  we  initialized  the  algorithm  at  an  arbitrary  signature  set  and  we  ob¬ 
served  its  convergence  to  a  new  signature  set  for  which  the  output  SINR  of 
each  user’s  RAKE/MMSE  filter  is  significantly  increased. 

To  further  improve  the  performance  of  a  communication  link  we  also 
considered  the  problem  of  interference  avoidance  at  the  transmitter  end  by 
designing  optimal  CDMA  spreading  codes.  In  particular,  we  designed  dou¬ 
bly  optimal  binary  spreading  code  sets  of  sequences  that  exhibit  minimum 
TSC  and  minimum  MSC  at  the  same  time  [16],  [17].  We  focused  on  the 
performance  of  sets  with  signatures  of  odd  length  and  we  derived  explicit 
closed-form  formulas  for  the  signature  cross-correlation  matrix  of  such  sets, 
its  eigenvalues,  and  its  inverse.  Based  on  these  formulas,  we  were  able  to 
obtain  analytical  expressions  for  the  BER  of  decorrelating  filters,  the  (max¬ 
imum)  output  SINR  of  MMSE  filters,  and  the  total  asymptotic  efficiency  of 
synchronous  code  division  multiplexing  systems.  The  calculated  across-the- 
board  superiority  of  the  “4m+l”  signature  sets  is  intriguing.  Combined  with 
the  indicated  similar  to  Gold  sets  performance  under  asynchronous  trans¬ 
missions,  this  strong  proven  superiority  for  (near-)synchronous  links  of  the 
“4m+r’  sets  questions  seriously  the  status  quo  and  the  popularity  of  Gold 
sets  (or  any  other  “4m— 1”  doubly  optimal  candidate  set  to  that  respect).  Re¬ 
gretfully,  neither  the  “4m+l”  codes  nor  the  Gold  codes  have  what  we  might 
call  satisfactory  performance  for  heavy-loaded  asynchronous  transmissions. 

Finally,  we  developed  a  new  multiuser  detection  algorithm  in  the  context 
of  asynchronous  AWGN  multiple-access  channels  [18], [19].  A  decorrelating 
or  MMSE/AV  multiuser  filter  is  used  as  a  pre-processor  that  provides  ini¬ 
tial  decisions  and  reliability  measurements  based  on  which  an  ordered  error 
pattern  sequence  of  tunable  length  is  formed.  The  error  pattern  sequence 
is  followed  to  its  end  and  the  most  likely  bit  vector  among  all  visited  op- 
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tions  is  returned.  When  the  length  of  the  pattern  sequence  is  of  the  order 
of  the  number  of  bits  to  be  detected,  the  additional  imposed  computational 
cost  compared  to  straight  decorrelating  or  MMSE/AV  detection  is  rather 
insignificant.  Still,  we  saw  that  this  proposed  multiuser  detection  algorithm 
maintains  near-ML  bit-error-rate  performance  over  the  whole  studied  SNR 
range  of  interest.  There  is  strong  resemblance  between  this  scheme  and  “ef¬ 
ficient”  decoding  algorithms  for  binary  linear  block  codes  [20]. 
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Data  Record  Adaptive  Filtering:  The  Auxiliary-Vector  Algorithm,  Digital 
Signal  Processing  12  (2002)  193-222. 

Based  on  statistical  conditional  optimization  criteria,  we  developed  an 
iterative  algorithm  that  starts  from  the  matched  filter  (or  constraint  vec¬ 
tor)  and  generates  a  sequence  of  filters  that  converges  to  the  minimum 
variance  distortionless  response  (MVDR)  solution  for  any  positive  defi¬ 
nite  input  autocorrelation  matrix.  Computationally  the  algorithm  is  a  sim¬ 
ple  recursive  procedure  that  avoids  explicit  matrix  inversion,  decomposi¬ 
tion,  or  diagonalization  operations.  When  the  input  autocorrelation  matrix 
is  replaced  by  a  conventional  sample-average  (positive  definite)  estimate, 
the  algorithm  effectively  generates  a  sequence  of  MVDR  filter  estimators: 

The  bias  converges  rapidly  to  zero  and  the  covariance  trace  rises  slowly 
and  asymptotically  to  the  covariance  trace  of  the  familiar  sample  matrix 
inversion  (SMI)  estimator.  For  short  data  records,  the  early  nonasymp- 
totic,  elements  of  the  generated  sequence  of  estimators  offer  favorable 
bias-covariance  balance  and  are  seen  to  outperform  in  mean-square  es¬ 
timation  error  constraint-LMS,  RLS-type,  and  orthogonal  multistage  de¬ 
composition  estimates  (also  called  nested  Wiener  filters)  as  well  as  plain 
and  diagonally  loaded  SMI  estimates.  The  problem  of  selecting  the  most 
successful  (in  some  appropriate  sense)  filter  estimator  in  the  sequence  for 
a  given  data  record  is  addressed  and  two  data-driven  selection  criteria  are 
proposed.  The  first  criterion  minimizes  the  cross-validated  sample  aver¬ 
age  variance  of  the  filter  estimator  output.  The  second  criterion  maximizes 
the  estimated  J-divergence  of  the  filter  estimator  output  conditional  dis¬ 
tributions.  Illustrative  interference  suppression  examples  drawn  from  the 
communications  literature  are  followed  throughout  this  presentation. 

©  2002  Elsevier  Science  (USA) 
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INTRODUCTION 

Minimum  variance  distortionless  response  (MVDR)  filtering  refers  to  the 
problem  of  identifying  a  linear  filter  that  minimizes  the  variance  at  its  output, 
while  at  the  same  time  the  filter  maintains  a  distortionless  response  toward  a 
specific  input  vector  direction  of  interest.  If  r  is  a  random,  zero  mean  without 
loss  of  generality,  complex  input  vector  of  dimension  L,  r  €  CL,  that  is  processed 
by  an  L-tap  filter  w  e  CL,  then  the  filter  output  variance  is  wwRw,  where 
R  =  E{ rrH]  is  the  input  autocorrelation  matrix  (£{•}  denotes  the  statistical 
expectation  operation  and  xH  denotes  the  Hermitian,  that  is,  the  transpose 
conjugate  of  x).  The  MVDR  filter  minimizes  wwRw  and  simultaneously  satisfies 
wH\  —  1,  or  more  generally  wH\  =  p  e  C,  where  v  e  CL  is  the  signal  vector 
direction  to  be  protected.  In  this  setup,  MVDR  filtering  is  a  standard  linear 
constraint  optimization  problem  and  a  conventional  Lagrange  multipliers 
procedure  leads  to  the  well-known  solution  [1,  2] 

WMVDR  =  P*  vtfR-lv  ’  (1) 

where  p*  denotes  the  conjugate  of  the  desired  response  wwv  =  p.  MVDR  fil¬ 
tering  has  been  used  extensively  in  unsupervised  signal  processing  applications 
where  a  desired  scalar  filter  output  d  e  C  cannot  be  identified  or  cannot  be  as¬ 
sumed  available  for  each  input  r  €  (for  example,  in  radar  and  array  process¬ 
ing  problems  where  the  constraint  vector  v  is  usually  referred  to  as  the  target  or 
look  direction  of  interest).  We  may  also  observe  the  close  relationship  between 
the  MVDR  filter  and  the  minimum  mean  square  error  (MMSE)  or  Wiener  filter. 
Indeed,  if  the  constraint  vector  v  is  chosen  to  be  the  statistical  cross-correlation 
vector  between  the  desired  output  d  and  the  input  vector  r,  that  is  if  v  =  E{rd*}, 
then  the  MVDR  and  MMSE  filters  become  scaled  versions  of  each  other,  cR_1v, 
c  e  C.  For  this  reason,  in  the  rest  of  the  paper  we  will  use  the  term  MMSE- 
MVDR  filter  to  refer  to  either  filter. 

In  this  article,  first  we  present  an  iterative  algorithm  for  the  calculation  of  the 
MMSE-MVDR  vector  Wmmse/mvdr  in  (D-  The  algorithm  is  a  noninvasive  pro¬ 
cedure  where  no  explicit  matrix  inversion-eigendecomposition-diagonalization 
is  attempted.  The  MMSE-MVDR  computation  algorithm  creates  a  sequence  of 
filters  w„,n  =  0, 1,2,...,  that  begins  from  wo  =  (p*/||v|[2)v  and  converges  to  the 
MMSE-MVDR  filter  (woo  =  wmmse/mvdr)-  At  each  step  n  =  1, 2, . . . ,  w„  is  given 
as  a  simple,  direct  function  of  R,  v,  and  w„_i. 
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The  development  of  the  iterative  algorithm  (which  we  call  the  auxiliary- 
vector  algorithm  for  reasons  that  will  become  apparent)  is  founded  solely  on 
statistical  signal  processing  principles.  The  motivation  behind  its  development 
is  adaptive  signal  processing  where  the  input  autocorrelation  matrix  R  is 
assumed  unknown  and  it  is  sample-average  estimated  by  a  data  record  of  M 
points,  ri,r2, 

ZrmT%-  (2) 

m= 1 

When  R  is  substituted  by  R(M)  in  the  recursively  generated  sequence 
of  filters  w„,  n  =  0, 1,2, . . the  corresponding  filter  estimators  wn(M),  n  = 
0,1,2,...,  offer  the  means  for  effective  control  over  the  filter  estimator  bias 
versus  (co-)variance  trade-off  [3].  Starting  from  the  0- variance,  high-bias  (for 
nonwhite  inputs)  wo(M)  =  (p*/||v||2)v  estimate,  we  can  go  all  the  way  up  to 
the  unbiased,  yet  high- variance  for  small  data  record  sizes  M ,  Woo(AU  estimate 
and  anywhere  in  between,  wn(Af),  1  <  n  <  oo.  As  a  result,  adaptive  filters  from 
this  newly  developed  class  can  be  seen  to  outperform  in  expected  norm-square 
estimation  error,  £{||w(M)  -  wmmse/mvdrII2}*  (constraint-)  LMS  [4],  sample 
matrix  inversion  (SMI)  [5]  with  or  without  diagonal  loading  [6],  RLS-type  [7,  8], 
and  orthogonal  multistage  decomposition  (also  called  nested  Wiener)  [9,  10] 
adaptive  filter  implementations.  It  is  worth  mentioning  that  the  familiar  trial- 
and-error  tuning  to  problem  and  data-record-size  specifics  of  the  real-valued 
LMS  gain  or  RLS  inverse  matrix  initialization  constant  or  SMI  diagonal  loading 
parameter  that  plagues  field  practitioners  is  now  replaced  by  an  integer  choice 
of  one  of  the  recursively  generated  filters. 

The  problem  of  selecting  the  best  (in  some  appropriate  sense)  filter  estimator 
in  the  sequence  for  a  given  data  record  is  addressed  and  two  data-driven 
selection  criteria  are  proposed.  The  first  criterion  is  rather  general  and 
is  motivated  by  the  asymptotic  minimum  output  variance  property  of  the 
generated  sequence  of  filter  estimators.  In  particular,  for  a  given  data  record, 
we  select  the  filter  estimator  that  has  minimum  cross-validated  average  output 
variance  (energy).  The  second  rule  is  built  specifically  for  binary  antipodal 
(BPSK-type)  communication  signals  and  is  related  to  the  objective  of  achieving 
maximum  stochastic  distance  between  the  two  conditional  distributions  of  the 
filter  estimator  output.  Under  this  rule,  we  choose  the  filter  estimator  in  the 
sequence  that  exhibits  maximum  estimated  J-divergence  of  the  conditional 
output  distributions.  We  pursue  and  analyze  both  supervised  and  unsupervised 
(blind)  implementations  of  this  criterion.  Illustrative  case  studies  drawn  from 
the  code  division  multiple  access  (CDMA)  communications  literature  are 
followed  throughout  this  article. 

The  rest  of  the  article  is  organized  as  follows.  In  Section  2  we  present  the 
basic  algorithmic  development  and  analysis  results.  Filter  estimation  issues  are 
discussed  in  Section  3.  The  two  data-driven  criteria  for  the  selection  of  a  filter 
estimator  from  the  generated  sequence  are  developed  in  Section  4.  In  Section  5 
we  examine  the  quality  of  the  proposed  criteria  through  simulations.  A  few 
concluding  remarks  are  given  in  Section  6. 


196 


Digital  Signal  Processing  Vol.  12,  Nos.  2,3,  April  and  July  2002 


2.  ALGORITHMIC  DEVELOPMENTS  AND  CONVERGENCE  ANALYSIS 


For  a  given  constraint  vector  veCL  consider  the  set  of  filters  £>  =  {we  CL: 
w  =  (p*/||v||2)v  +  u,  u  e  CL,  and  vwu  =  0).  V  is  the  class  of  all  filters  w  in  CL 
that  have  a  given  response  p  in  v;  that  is,  wwv  =  p.  In  this  section  we  develop 
an  iterative  algorithm  for  the  computation  of  the  u  component  of  the  MMSE- 
MVDR  filter. 

Algorithmic  designs  that  focus  on  the  MMSE-MVDR  filter  part  u  that  is 
orthogonal  to  the  constraint  vector,  or  look,  direction  v  have  been  widely 
pursued  in  the  array  processing  literature  and  have  been  known  as  generalized 
sidelobe  cancelers  (GSC)  [11]  or  partially  adaptive  beamformers  [12].  Recent 
developments  have  been  influenced  by  principal  component  analysis  reduced- 
rank  processing  principles  [13].  In  general,  the  MMSE-MVDR  filter  part  u 
(uwv  =  0)  has  been  approximated  by  uLxi  ~  -BLx(i_i)T(/,_1)x/,w^,  where  B 
is  the  so-called  blocking  matrix  that  satisfies  Bwv  =  Ot_i  (B  is  a  full  column- 
rank  matrix  that  can  be  derived  by  Gram-Schmidt  orthogonalization  of  an 
L  x  L  orthogonal  projection  operator  such  as  I  -  ww/||v||2,  where  I  is  the 
identity  matrix),  T  is  the  rank  reducing  matrix  with  1  <  p  <  L  -  1  columns 
to  be  designed,  and  wGSC  is  the  MS-optimum  vector  of  weights  of  the  p  columns 
of  T  (wGSC  =  (p*/||v||2)[T//BwRBT]-1TwBHRv  [12]).  In  [14]  and  [15]  the  p 
columns  of  T  were  chosen  to  be  the  p  maximum  eigenvalue  eigenvectors  of  the 
blocked  data  autocorrelation  matrix  BWRB.  If,  however,  the  columns  of  T  have 
to  be  eigenvectors  of  BWRB  (there  is  no  documented  technical  optimality  to  this 
approach),  then  the  best  way  to  choose  them  in  the  minimum  output  variance 
p-rank  approximation  sense  was  presented  in  [16]:  Select  the  p  eigenvectors 
q,  of  BHRB,  with  corresponding  eigenvalues  A, ,  that  maximize  |vwRBq,j2/A,  , 
i  =  1, . . . ,  p.  This  design  algorithm  was  called  cross-spectral  metric  reduced- 
rank  processing  in  [17].  A  different  approach  from  a  different  point  of  view 
is  described  in  this  work.  A  conditional  statistical  optimization  procedure  is 
shown  to  offer  the  means  for  exact  computation  of  u  as  the  convergence  point 
of  an  infinite  series  of  the  form  -  X^Li  png„,  pn  e  TZ+,  g„  e  CL,  and  g«v  =  0, 
Vn  =  1, 2 . 

We  begin  the  algorithmic  developments  from  the  conventional  matched  filter 
(MF)  with  desired  response  wwv  =  p 


w0  = 


(3) 


which  is  MMSE-MVDR  optimum  for  white  CL  vector  inputs  (when  R  =  a2I, 
a  >  0).  We  recall  that,  w.l.o.g.  and  for  notational  simplicity,  we  assume  through¬ 
out  this  presentation  that  the  input  vectors  r  €  C ^  are  zero  mean.  Next,  we 
incorporate  in  wo  an  “auxiliary”  vector  component  that  is  orthogonal  to  v  and 
we  form  (Fig.  1) 


P* 

l|v||2 


v-p  lgl. 


wi  =w0  -pigi  = 


(4) 
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FIG.  1.  Block  diagram  representation  of  the  iteratively  generated  sequence  of  filters 
Wo,Wi,W2, - 


where  gx  e  CL  -  {0},  /zx  <=  C,  and  gfv  =  0.  We  assume  for  a  moment  that 
the  orthogonal  auxiliary  vector  gx  is  arbitrary  but  nonzero  and  fixed  and  we 
concentrate  on  the  selection  of  the  scalar  fi\ .  The  value  of  fi\  that  minimizes 
the  variance  of  the  output  of  the  filter  wx  can  be  found  by  direct  differentiation 
of  £{|wfr|2}  or  simply  as  the  value  that  minimizes  the  MS  error  between 
w^r  =  (p/||v||2)vHr  and  /xjgf  r.  This  is  essentially  a  scalar  version  of  the  GSC 
weight  determination  problem  and  we  present  the  solution  in  the  form  of  a 
proposition  [18]: 


PROPOSITION  1.  The  scalar  that  minimizes  the  variance  at  the  output  of 
wx  or  equivalently  minimizes  the  MS  error  between  w^r  =  (p/||v||2)vHr  and 
Mi  gfr  is 


Ml  = 


gfRwp 
gf  Rgi  ’ 


(5) 


where  R  =  E[rrH}  is  the  input  autocorrelation  matrix. 


Since  gi  is  set  to  be  orthogonal  to  v,  (5)  shows  that  if  the  vector  Rwo 
happens  to  be  on  v  (that  is,  Rw0  =  (vHRw0)/(||v||2)v  or  equivalently  (I  - 
(wh/||v[|2)Rw0  =  0),  then  fi\  =  0.  Indeed,  if  Rw0  =  (vwRw0/||v||2)v  then  w0 
is  already  the  MMSE-MVDR  filter.  To  avoid  this  trivial  case  and  continue 
with  our  developments,  we  suppose  that  Rwo  ^  (vHRwo/||v||2)v.  By  inspection, 
we  also  observe  that  for  the  MS-optimum  value  of  p\  the  product  jxigi  = 
(gj^Rwo/gf  Rgi)gi  is  independent  of  the  norm  of  gi.  Hence,  so  is  wi.  At  this 
point,  we  decide  to  choose  the  auxiliary  vector  gi  as  the  normalized  vector  that 
maximizes  the  magnitude  of  the  cross-correlation  between  w^r  =  (p/||v||2)vHr 
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and  gfr,  under  the  constraint  that  gf v  =  0  and  g^gi  =  1: 

gi  =  argmax|£{w^r(gwr)*}|  =  argmax|w^Rg| 

subject  to  gwv  =  0  and  gHg  =  1.  (6) 

For  the  sake  of  mathematical  accuracy,  we  note  that  both  the  criterion 
function  |w^Rg|  to  be  maximized  and  the  orthogonality  constraints  are 
phase  invariant.  In  other  words,  if  g!  satisfies  (6)  so  does  giej<s>  for  any 
phase  <p.  Without  loss  of  generality,  to  avoid  any  ambiguity  in  our  presentation 
and  to  have  a  uniquely  defined  auxiliary  vector,  we  seek  the  one  and  only 
auxiliary  vector  that  satisfies  (6)  and  places  the  cross-correlation  value  on  the 
positive  real  line  (w^Rg  >  0).  This  constraint  optimization  problem  was  first 
posed  and  solved  in  [19]  where  the  filter  wi  in  (4)  was  used  for  multiple 
access  interference  suppression  in  multipath  CDMA  communication  channels. 
Intuitively,  the  maximum  magnitude  cross-correlation  criterion  as  defined  in  (6) 
strives  to  identify  the  auxiliary  vector  orthonormal  to  v  that  can  capture  the 
most  interference  present  in  wfl r.  The  solution,  derived  through  conventional 
Lagrange  multipliers  optimization,  is  given  below. 

Proposition  2.  Suppose  that  (I  -  wH/||v||2)Rw0  #  0  (w0  ^  wmmse/mvdr). 
Then,  the  auxiliary  vector 


gi  = 


Rw0  - 

||Rw0  - 


v"Rw° 

Imp 


vwRwn 

"Imp 


(7) 


maximizes  the  magnitude  of  the  cross-correlation  between  w^r  =  (p/||v||2)vwr 
and  gf  r,  Iw^Rgi  |,  subject  to  the  constraints  gfv  =  0  and  gf  gr  =  1.  In  addition, 
wq  Rgi  is  real  positive  (wf  Rgi  >  0). 

So  far  we  have  defined  w0  in  (3)  and  wi  in  (4)  with  gi  and  ui  given  by  (7) 
and  (5),  respectively.  The  iterative  algorithm  for  the  generation  of  an  infinite 
sequence  of  filters  w0,wi,w2,...  is  already  taking  shape.  Formally,  we  just 
need  to  specify  the  inductive  step.  Assuming  that  the  filter  w„  =  (p*/||v||2)v  - 
E;=i  Migi  has  been  identified  for  some  n  >  1  and  wn  we  argue 

as  in  Propositions  1  and  2  and  we  define 


wn+l  =  W„  -Mn+lgn+l, 


(8) 


where 


gn+1  = 


RW"~1^V 
lRw"  -  wHI 


(9) 


is  the  orthonormal  auxiliary  vector  (with  respect  to  v)  that,  given  wn ,  maximizes 
conditionally  the  cross-correlation  magnitude  |£{wfr(gf+1r)*}|  =  |wfRgn+1| 
and 


g"+iRw* 

gn+l^gn+l 


dn+1- 


(10) 
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is  the  scalar  that  minimizes  the  MS  error  between  w^r  and  /t*+1g^flr 
(minimizes  2s{|w^+1r|2}). 

It  is  important  to  note  that,  while  the  generated  auxiliary  vectors  gi,  g2,  •  •  • 
are  all  constrained  to  be  orthogonal  to  v,  orthogonality  among  the  auxiliary 
vectors  is  not  imposed  [20,  21].  This  is  in  sharp  contrast  to  previous  work  that 
involved  filtering  with  up  to  L  —  1  orthogonal  to  each  other  and  to  v  vectors 
[22-24],  where  L  is  the  data  input  vector  dimension.  We  observe,  however, 
that  successive  auxiliary  vectors  generated  by  the  above  recursive  conditional 
optimization  procedures  (8)— (10)  do  come  up  orthogonal:  g^g«+i  =  0,  Vn  = 
1, 2, 3, . . .  (while  g*gm  ^  0,  Vn, m,  |n  -  m|  #  1).  For  completeness  purposes,  we 
present  this  observation  below  in  the  form  of  a  lemma. 

LEMMA  1.  Successive  auxiliary  vectors  generated  through  (8)— (10)  are  or¬ 
thogonal:  g^gn+i  =  0,  n  —  1, 2, 3,  —  However,  gffg m  ^  0,  Wn,  m,  \n  -  m  \  ^  1. 

The  algorithm  is  summarized  in  Fig.  2.  The  conceptual  simplicity  of  the 
conditional  statistical  optimization  process  led  to  a  computationally  simple 
recursion.  In  Fig.  2  we  chose  to  drop  the  unnecessary,  as  previously  explained, 
normalization  of  the  auxiliary  vectors  and  we  also  factorized  their  numerator  to 
make  the  orthogonal  projection  operator  apparent.  Formal  convergence  of  the 
filter  sequence  wo,  wi,  W2, ...  to  the  MMSE-MVDR  filter  p*R“1v/vwR“1v  is 
established  by  the  following  theorem.  The  proof  can  be  found  in  [21]. 

THEOREM  1.  Let  R  he  a  Hermitian  positive  definite  matrix.  Consider  the 
iterative  algorithm  of  Fig.  2. 


input: 


initialization 
wr 


Auxiliary-Vector  Algorithm 


Autocovariance  matrix  R,  constraint  vector  v, 
desired  response  wHv  =  p  . 


£  v 

llvll2  • 


(Iterative  computation: 

For  n=1 , 2,  ...  do 
begin 

9n  :=  (  I  "  jjyjp  )  R  W  n-1 


if  fln  =  0  then  EXIT 

.  gJRw^ 

•  JFR97 

Wn  :=  Wn.T-Jinfln 


end 

(Output: 

Filter  sequence  wn,  w1t  wP, 


FIG.  2.  The  algorithm  for  the  iterative  generation  of  the  filter  sequence  w0 ,  Wi ,  w2 
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(i)  The  generated  sequence  of  auxiliary -vector  weights  {pn},  n  =  1, 2, . . is 
real-valued ,  positive ,  and  bounded, 

n  1  1 

°<T -  <Hn<- - ,  n  =  1,2 .  (11) 

Amax  Amin 

where  Xmax  and  km[n  are  the  maximum  and  minimum,  correspondingly, 
eigenvalues  of  R 

(ii)  The  sequence  of  auxiliary  vectors  [g„),  n  -  1,2,...,  converges  to  the  0 
vector: 

nlimgn=°.  (12) 

(iii)  The  sequence  of  auxiliary-vector  filters  (w„),  n  =  1,2,...,  converges  to 
the  MMSE-MVDR  filter: 


lim  w„  = 

n-+oc 


*  R_1v 
P  vwR-1v' 


(13) 


We  conclude  this  section  with  an  illustration.  We  draw  a  signal  model 
example  from  the  direct-sequence  code  division  multiple  access  (DS-CDMA) 
communications  literature  and  we  assume  a  synchronous  system  where  the 
input  signal  vector  r  e  TZL  is  given  by 


K 

r=J2-/Ekbksk+n.  (14) 

*= l 

In  this  setup,  K  denotes  the  total  number  of  signals  (users)  present  and  each 
signal  is  defined  through  an  L-dimensional,  normalized,  binary-antipodal  vector 
waveform  (or  user  signature)  s*,  k  =  1, 2, . . . ,  K.  The  signature  vector  dimension 
L  is  usually  referred  to  as  the  system  spreading  gain.  With  respect  to  the  Jfcth 
user  signal,  Ek  is  the  received  signal  energy  and  bk  e  {-1,  +1}  is  the  information 
bit  modeled  as  a  random  variable  with  equally  probable  values  and  assumed 
to  be  statistically  independent  from  all  other  user  bits  bj,  j  ±  k.  Additive 
white  Gaussian  noise  contributions  are  accounted  for  by  n  with  autocorrelation 
matrix  £{nnT)  =  a2I l*l  (xt  denotes  the  transpose  of  x).  With  this  notation 
and  normalized  user  signatures,  the  signal-to-noise  ratio  of  the  Arth  user  signal 
is  defined  by  SNR*  =  101og10  £*/<r2  dB,  k  =  1, 2, . . . ,  K. 

MMSE-MVDR  filtering  for  DS-CDMA  type  problems  has  attracted  significant 
interest  [25-29].  If  we  wish  to  recover  the  information  bits  of,  say,  user  1, 
then  all  other  signals  constitute  multiple-access  interference  and  the  MMSE- 
MVDR  filter  is  built  with  constraint  vector  v  =  s*,  desired  response  wTsi  =  1, 
and  autocorrelation  matrix  R  =  J^Li  EkaksJ  +  <r2I.  We  choose  L  =  32,  K  =  13, 
and  we  draw  an  arbitrary  set  of  signatures  Si,S2, ...,Si3.  For  purposes  of 
completeness  in  presentation,  the  exact  signature  assignment  is  given  in  the 
Appendix.  We  fix  the  SNR  of  the  user  of  interest  at  SNR*  =  12  dB  while  the 
interferes  k  =  2, . . . ,  13  are  at  SNR2-5  =  10  dB,  SNR^g  =  12  dB  and  SNR10-i3  = 
14  dB.  Figure  3  shows  how  the  sequence  of  filters  wq,  w*,  . . .  generated  by  the 
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FIG.  3.  Convergence  of  the  sequence  of  filters  w„,  n  =  0, 1, 2, . . . ,  to  the  MMSE-MVDR  solution 
for  the  signal  model  example  in  (14). 


algorithm  in  Fig.  2  converges  to  the  MMSE— MVDR  solution.  The  convergence 
is  captured  in  terms  of  the  norm-square  metric  ||w„  -  wmmse/mvdrII2  as  a 
function  of  the  iteration  step  (index  of  the  AV  filter  in  the  sequence  or  number 
of  auxiliary  vectors  used)  n. 


3.  FILTER  ESTIMATION 


Consider  a  constraint  vector  v  and  a  Hermitian  positive  definite  autocor¬ 
relation  matrix  R  of  an  input  vector  reC1.  Assume  that  R  is  in  fact  un¬ 
known  and  it  is  sample-average  estimated  from  a  data  record  of  M  points: 
R(M)  =  (1/M)  Y.m= l  rmrm  • For  Gaussian  inputs,  R(M)  is  a  maximum-likelihood 
(ML),  consistent,  unbiased  estimator  of  R  [3,  30].  For  a  large  class  of  multi¬ 
variate  elliptically  contoured  input  distributions  that  includes  the  Gaussian, 
if  M  >  L  then  R(M)  is  positive  definite  (hence  invertible)  with  probability  1 
(w.p.  1)  [31-33].  Then,  Theorem  1  in  Section  2  shows  that 


w„(M)- 


>  Woo (M)  =  p* 


[R(M)r 


\H  [R(M)] 


-U 


(15) 


where  Woo(M)  is  the  widely  used  MMSE-MVDR  filter  estimator  known  as  the 
SMI  filter  [5], 

The  output  sequence  begins  from  wo(M)  =  (p*/||v||2)v,  which  is  a  0-variance, 
fixed-valued  estimator  that  may  be  severely  biased  (wo(M)  =  (p*/||v||2)v  ^ 
wjjmse/mvdr)  unless  R  =  cr2I,  for  some  o  >  0.  In  the  latter  trivial  case,  wo(M)  is 
already  the  perfect  MMSE-MVDR  filter.  Otherwise,  the  next  filter  estimator  in 
the  sequence,  wi(M),  has  a  significantly  reduced  bias  due  to  the  optimization 
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procedure  employed,  at  the  expense  of  nonzero  estimator  (co-)  variance. 
As  we  move  up  in  the  sequence  of  filter  estimators  w „{M),  n  =  0,1,2,..., 
the  bias  decreases  rapidly  to  zero2  while  the  variance  increases  slowly  to 
the  SMI  (WooCAf))  levels  (cf.  (15)).  To  quantify  these  remarks,  we  plot  in 
Fig.  4  the  norm-square  bias  ||£{w„(Af)}  -  wmmse/mvdrII2  and  the  trace  of  the 
covariance  matrix  £{[w„(M)  -  £{wn(M))][w„(A/)  -  £{w„(M)}]w}  as  a  func¬ 
tion  of  the  iteration  step  n  for  the  signal  model  example  of  Fig.  3  and 
data  record  size  M  =  256.  Bias  and  cov-trace  values  are  calculated  from 
100000  independent  filter  estimator  realizations  for  each  iteration  point  n. 
Formal,  theoretical  statistical  analysis  of  the  generated  estimators  w„(Af), 
n  =  0,1,2, ...,  is  beyond  the  scope  of  this  presentation.  We  do  note,  how¬ 
ever,  that  for  multivariate  elliptically  contoured  input  distributions,  an  an¬ 
alytic  expression  for  the  covariance  matrix  of  the  SMI  estimator  Woo  (M) 
can  be  found  in  [33]:  £{[Woo(M)  -  £{w00(M)}](w00(A/)  -  £{Woo(W)}]w}  = 
[|p|2£(vwR  1v)(A/  —  L  +  1)](R_1  —  R-1wHR_1/vffR_1v).  Since  under  these 
input  distribution  conditions  wcc(M)  is  unbiased,  the  trace  of  the  covariance 
matrix  is  the  MS  filter  estimation  error.  It  is  important  to  observe  that  the  co- 
variance  matrix  and,  therefore,  the  MS  filter  estimation  error  depend  on  the 
data  record  size  M  and  the  filter  length  L,  as  well  as  the  specifics  of  the  signal 
processing  problem  at  hand  (R  and  v).  It  is  also  important  to  note  that  for  the 
CDMA  signal  model  example  in  (14)  the  input  is  Gaussian-mixture  distributed. 
Therefore,  the  analytic  result  in  [33]  is  not  directly  applicable  and  can  only  be 
thought  of  as  an  approximation  (a  rather  close  approximation  as  we  concluded  in 
our  studies).  In  any  case,  from  the  results  in  Fig.  4  for  M  =  256,  we  see  that  the 


FIG.  4.  Norm-square  bias  and  covariance  trace  for  the  sequence  of  estimators  w „(M), 
»  =  0, 1 . The  signal  model  is  as  in  Fig.  3  and  M  =  256. 


The  SMI  estimator  is  unbiased  for  multivariate  elliptically  contoured  input  distributions 
[33,  34]:  FfWooiA/)]  —  ^mmse/mvdr  =  p*R-1v/vwR-1v. 
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(a) 


(b) 


FIG.  5.  MS  estimation  error  for  the  sequence  of  estimators  w„  (M),  n  =  0,1,  —  Data  record  size 
(a)  M  —  256,  (b)  M  =  2048. 


estimators  wi(M),  W2(M), . . . ,  up  to  about  W20 (M )  are  particularly  appealing.  In 
contrast,  the  estimators  w  n{M)  for  n  >  20  do  not  justify  their  increased  cov-trace 
cost  since  they  have  almost  nothing  to  offer  in  terms  of  further  bias  reduction. 

The  mean-square  estimation  error  expression  E{\\wn(M )  -  wmmse/mvdrII2} 
captures  the  bias-variance  balance  of  the  individual  members  of  the  estimator 

sequence  w n(M),  n  =  0, 1, 2, _ In  Fig.  5  we  plot  the  MS  estimation  error  as 

a  function  of  the  iteration  step  (index  of  AV  filter  in  the  sequence  or  number 
of  auxiliary  vectors)  n  for  the  case  study  in  Fig.  4,  for  M  =  256  (Fig.  5a) 
and  M  —  2048  (Fig.  5b).  As  a  reference,  we  also  include  the  MS -error  of  the 
constraint-LMS  estimator  [4,  35] 

/  x\H 

WLMS(w)=^-|^jj2 

m  =  (16) 


)  [wlms(w  -  1)  -  fJ.rmr%wiMS(m  -  1)]  + 


with  wlms(0)  =  (/>7llv||2)v  and  some  /x  >  0,  and  the  RLS  estimator  [7,  8]  with 
matrix-inversion-lemma-based  R-1  estimation: 


R  1  (m)  =  R  1(m  —  1) — 


R  x(w-l)rmr"R  1(m-  1) 
1  +  rm  R_1(m  —  l)rm 


m  =  1 . M,  (17) 


with  R_1(0)  =  (l/eo)I  for  some  eo  >  0.  Theoretically,  it  is  known  that  the 
LMS  gain  parameter  /x  >  0  [36]  has  to  be  less  than  1/(2  •  A.^ked),  where 
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FIG.  6.  MS  estimation  error  versus  number  of  auxiliary  vectors  n  and  sample  support  M. 


^blocked  jg  the  maxjmum  eigenvalue  of  the  blocked-data  autocorrelation  matrix 
(I  —  ww/||v||2)R(I  —  ww/||v||2).  While  this  is  a  theoretical  upper  bound, 
practitioners  are  well  aware  that  empirical,  data-dependent  optimization  or 
tuning  of  the  LMS  gain  /z  >  0  or  the  RLS  initialization  parameter  eo  >  0 
[37]  is  necessary  to  achieve  acceptable  performance  (in  our  study  we  set  /z  = 
1/(200  *  ^maxked)  £o  =  20,  respectively).  This  data  specific  tuning  frequently 
results  in  misleading,  overoptimistic  conclusions  about  the  short  data  record 
performance  of  the  LMS  and  RLS  algorithms.  In  contrast,  when  the  filter 
estimators  w„  generated  by  the  algorithm  of  Fig.  2  are  considered  instead, 
tuning  of  the  real- valued  parameters  /z  and  €o  is  virtually  replaced  by  an  integer 
choice  among  the  first  several  members  of  the  {wn}  sequence.  Adaptive,  data- 
dependent  criteria  for  the  selection  of  the  most  appropriate  AV  filter  in  the 
sequence  for  a  given  data  record  are  developed  in  the  next  section.  In  Fig.  5a,  for 
M  —  256  all  estimators  w„  from  n  =  2  up  to  about  n  =  55  outperform  in  MS-error 
their  RLS,  LMS,  and  SMI  (Woo)  counterparts.  Ws  (n  =  8  auxiliary  vectors)  has 
the  least  MS-error  of  all  (best  bias-variance  trade-off).  When  the  data  record 
size  is  increased  to  M  =  2048  (Fig.  5b),  we  can  afford  more  iterations  (more 
auxiliary  vectors)  and  W13  offers  the  best  bias-variance  trade-off  (lowest  MS- 
error).  All  filter  estimators  wn  for  n  >  8  outperform  the  LMS,  RLS,  and  SMI 
(Woo)  estimators.  For  such  large  data  record  sets  (M  =  2048),  the  RLS  and  the 
SMI  (Wqo)  MS -errors  are  almost  identical.  Figure  6  offers  a  3-dimensional  plot 
of  the  MS  estimation  error  as  a  function  of  the  number  of  auxiliary  vectors  n 
and  the  sample  support  M .  The  dark  line  that  traces  the  bottom  of  the  MS 
estimation  error  surface  identifies  the  best  number  of  auxiliary  vectors  for  any 
given  data  record  size  A/. 
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Number  of  auxiliary  vectors  n 

FIG.  7.  MS  estimation  error  studies  for  (a)  diagonally  loaded  SMI,  (b)  multistage,  and 
(c)  auxiliary-vector  estimators  (M  =  60). 


An  alternative  bias-variance  trading  mechanism  through  read-valued  tuning 
is  the  diagonally  loaded  (DL)  SMI  estimator  [6] 


WDL-SMl(A)  = 


»  [R(AQ  +  Al]~1v 

P  vH[R(M)  + AIJ-V 


(18) 


where  A  >  0  is  the  diagonal  loading  parameter.  We  observe  that  Wdl-Smi(A  =  0) 
is  the  regular  SMI  estimator,  while  lim  wdLtSmi(A)  =  (p*/||v||2)v  which  is  the 

A — >oo 

properly  scaled  matched  filter.  In  Fig.  7a  we  plot  the  MS  estimation  error  of  the 
DL-SMI  estimator  as  a  function  of  the  diagonal  loading  parameter  A  (Af  =  60). 
We  identify  the  best  possible  diagonal  loading  value  A  2^  3.45  (at  significant 
computational  cost)  and  in  Fig.  7c  we  compare  the  best  DL-SMI  estimator 
against  the  AV  estimator  sequence  for  which  no  diagonal  loading  is  performed. 
Interestingly,  the  AV  estimators  wn  from  n  —  4  to  7  outperform  in  MS-error  the 
best  possible  DL-SMI  estimator  (A  2^  3.45). 

Finally,  a  finite  set  of  L  filter  estimators  with  varying  bias-covariance 
balance  can  be  obtained  through  the  use  of  the  orthogonal  multistage  filter 
decomposition  procedure  in  [9,  10]  (the  resulting  filters  have  been  also  referred 
to  as  nested  Wiener  filters).  It  can  be  shown  theoretically  that  the  l- stage  filter, 
w/. stage,  0  <  l  <  L  •—  1,  is  equivalent  to  the  following  structure.  First,  change  the 
auxiliary-vector  generation  recursion  in  (9)  or  Fig.  2  to  impose  orthogonality 
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not  only  with  respect  to  the  constraint  vector  v  but  also  with  respect  to  all 
previously  defined  auxiliary  vectors  yi,y2,  •••,y„-i,  n<L-l: 


yn  =  ( I  - 


w 


H 


llv||2 


'fin*?' 

fcrlly,il2. 


Rw, 


n-1- 


(19) 


Next,  terminate  the  recursion  at  n  =  /,  0  <  l  <  L  -  1,  and  organize  the  / 
orthogonal  to  each  other  and  to  v  vectors  yi, . . . ,  y/  in  the  form  of  a  blocking 
matrix  Bix,  =  [yi,  y2 . y,].  Then, 


w/ -stage  — 


v-Btx/5/xi. 


(20) 


where 

fi  =  ^l[BwRB]-1BffRv  (21) 

is  the  MS  vector-optimum  (unconditionally  optimum)  set  of  weights  of  the 
vectors  yi, y2.  •  •  •  ,yi-  ®  In  the  context  of  MMSE-MVDR  filter  estimation  from 
a  data  record  of  size  M,  wo.stag e(M)  is  the  matched  filter  and  W(£_i).8tage(Af)  is 
the  SMI  estimator.  In  Fig.  7b  we  plot  the  MS  estimation  error  of  w/.stage(M) 
as  a  function  of  l, 0<1<L  —  1  =  31  ( M  =  60).  We  identify  the  best  multistage 
estimator  ( 1  =  3  stages)  and  in  Fig.  7c  we  compare  against  the  AV  estimator 
sequence.  We  see  that  all  AV  estimators  w„  from  n  =  3  to  8  outperform  in  MS- 
error  the  best  multistage  estimator  (1  =  3  stages).  Finally,  as  a  last  study,  in 
Fig.  8  we  plot  the  MS-error  of  the  A  =  3.45  DL-SMI  estimator  together  with  the 
MS-error  of  the  best  multistage  and  AV  estimators  over  the  data  support  range 
M  =  L/2  =  16  to  M  =  3L  =  96. 


4.  HOW  TO  CHOOSE  THE  NUMBER  OF  AUXILIARY  VECTORS 


In  this  section  we  present  two  data-driven  rules  for  the  selection  of  the 
number  of  auxiliary  vectors  n  [39].  The  first  rule  selects  the  AV  filter  estimator 
with  n  auxiliary  vectors  that  has  minimum  cross-validated  average  filter  output 
energy.  The  second  selection  rule  is  specific  to  BPSK  communications  receivers 
that  employ  a  sign  detector  at  the  output  of  the  linear  auxiliary-vector  filter. 
Details  are  given  below. 


*  Therefore,  the  multistage  filter  in  [9, 10)  is  identical  to  the  filter  wB  as  it  appears  in  [22-24],  The 
multistage  decomposition  algorithm  is  a  computationally  efficient  procedure  for  the  calculation  of 
this  filter  tailored  to  the  particular  structure  of  B"RB  (tridiagonal  matrix).  The  same  computational 
savings  can  be  achieved  by  the  general  forward  calculation  algorithm  of  Liu  and  Van  Veen  [38]  that 
returns  all  intermediate  stage  filters  along  the  way,  up  to  the  stage  of  interest  l  (total  computational 
complexity  of  order  0{(M  +  1)L2)),  The  AV  algorithm  in  Fig.  2  h««  computational  complexity 
0((M  +  n)L2)  where  n  is  the  desired  number  of  auxiliary  vectors.  Again,  all  intermediate  AV  filters 
are  returned.  Estimators  of  practical  interest  have  i «  M  or  n  «  M.  Therefore,  the  complexity  of  all 
such  algorithms  is  dominated  by  0(ML2)  which  is  required  for  the  computation  of  R(Af). 
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FIG.  8.  MS  estimation  error  for  the  best  multistage  and  AV  estimators  over  the  data  support 
range  M  =  1/2  =  16  to  M  =  3L  =  96.  The  MS  estimation  error  of  the  A  =  3.45  DL-SMI  estimator  is 
also  included  as  a  reference. 


4.1 .  Cross-Validated  Minimum  Output  Variance  Rule  (CV-MOV) 

Cross-validation  is  a  well-known  statistical  method  [40].  Here,  we  use  cross- 
validation  to  select  the  filter  parameter  of  interest  (number  of  auxiliary 
vectors  n)  that  minimizes  the  output  variance  which  is  estimated  based  on 
input  observations  that  have  not  been  used  in  the  process  of  building  the  filter 
estimator  itself. 

A  particular  case  of  cross-validation  that  we  use  in  this  work  is  the  leave-one- 
out  method.  The  following  criterion  defines  the  CV-MOV  AV  filter  estimator 
selection  process. 

CRITERION  1.  For  a  given  data  record  of  size  M,  the  cross-validated  mini¬ 
mum  output  variance  AV  filter  estimator  selection  rule  chooses  the  AV  filter  es¬ 
timator  w W1(M)  that  minimizes  the  cross-validated  sample  average  output  vari¬ 
ance  f  i.e.j 

n\  =argmin 

n 

where  (Af\m)  identifies  the  AV  filter  estimator  that  is  evaluated  from  the  available 
data  record  after  removing  the  mth  sample. 

It  may  be  important  to  emphasize  the  need  for  invoking  the  cross-validation 
technique  for  the  evaluation  of  the  sample-average  output  variance.  If  a  sample- 
average  evaluation  using  all  data  were  attempted,  then  the  selection  rule  would 
take  the  form  min{w^(M)R(M)w„(Af)},  where  R (Af)  =  (1/ A/)  3™* 

n  ^ 

w n(M)  is  given  by  the  algorithm  of  Fig.  2  with  R(M)  in  place  of  R.  Suchjnin- 
imization,  however,  would  result  in  n  =  oo  since  we  know  that  wjf  (M)R(Af) 


M 


wf  (M\m)rmr^w„(M\rn)  | 


(22) 


71=1 
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w,,(Af)  w^)(A/)R(A/)w°0(A/)  and  the  SMI  estimator  Woo(M)  achieves  mini- 

mum  sample  average  output  variance  w^(M)R(M)Wco(M)  (but  not  minimum 
true  output  variance  w£(A/)Rwoo(M),  of  course). 

4.2.  Output  J-divergence  Rule 

For  illustration  purposes  we  reconsider  the  BPSK  CDMA  signal  model 
example  in  (14).  The  output  J-divergence  rule  selects  the  AV  filter  estimator 
from  the  sequence  of  AV  estimators  that  maximizes  the  J-divergence  of  the 
Gaussian  approximated  conditional  filter-output  distributions  (conditioned  on 
the  transmitted  information  bit  b\  —  +1  or  b\  =  —1).  The  appropriateness  of 
such  a  criterion  as  well  as  implementation  details  are  presented  below. 

For  a  given  AV  filter  estimator  w„  (A/),  we  denote  by  pm  the  real  part 4  of  the 
filter  output  with  input  the  mth  data  vector  rm ,  m  =  1, 2, . . . ,  M , 

pm  =  Re[w"  (A/)rm]=  y£^Re[fci(m)w"(A/)si] 

K 

+  J^y/EkRe[bk(m)w^  (M)sk]  +  Re[w"(Af)nm],  (23) 

k=2 

where  the  information  bits  bk  (m ) ,  k  =  1,2. . . . ,  K ,  m  =  1, 2, ... ,  M,  are  assumed 
to  be  independent  identically  distributed  (i.i.d.)  with  equally  probable  outcomes 
and  nm  is  a  0-mean  complex  white  Gaussian  random  vector  with  autocovariance 
matrix  tr2I.  Then,  the  scalars  pm,  m  =  1,2, are  i.i.d.  with  common 
distribution  fp(x)  given  by 


/,(,)  -  1  f  exnf  "I*  -  ELl  VEReti.f  >«?(M)8,)|2 1 

’FW  2^(Sjp - /'  <24> 


where  b^\  i  —  1,2,...,  2* ,  is  the  bit  of  user  k  in  the  ith  bit-combination. 

Conditioned  on  the  transmitted  information  bit  of  the  user  of  interest,  user  1, 
the  pdf  of  the  filter  output  is  a  mixture  of  2K~l  Gaussian  distributions.  However, 
for  effective  interference  suppressive  filters  we  can  safely  approximate 
the  conditional  output  distribution  by  a  Gaussian  distribution  as  argued 
in  [41]  for  MMSE-MVDR  linear  filtering.  Under  this  approximation,  the 
filter  output  conditional  distributions  given  that  +1  or  -1  is  transmitted 
are  /i,„  ~  ’  a/2+v(w)]  and  fo,n  ~  Ar[—fi(n),  crf+N(n)],  respectively,  where 

p(n)  =  V£TRa[w"(M)8i]  and  af+N(n)  =  £*=2  Ek Re[w„w(A/)s*]2  +  o2||w„(Af)||2 
is  the  conditional  variance  due  to  multiple  access  interference  and  additive 
white  Gaussian  noise  (AWGN)  (the  index  “I+N”  denotes  comprehensively 
the  disturbance  contribution).  The  effect  of  the  above  approximation  on  the 
performance  of  the  output  J-divergence  selection  rule  will  be  examined  in 
Section  5. 

The  J-divergence  distance  fo,n)  between  the  distributions  /x,„( .)  and 

fo,n  (*)  is  defined  as  the  sum  of  the  Kullback-Leibler  (K-L)  distances  between 


4  While  the  signal  model  in  (14)  is  real- valued,  we  choose  to  carry  out  this  presentation  in  the 
more  general  context  of  complex  input  vectors  and  filters. 
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fi.n  and  /o,„ 


fo ,n)  =  D(fl,n.fo,«)  +  D(fo,n,  /l,„),  (25) 

where  the  K-L  distance  of  /i,„  from  /o,„  is  defined  by  D(/i,«,  /o,«)  =  fi,n(x) 
log(fi.„(x)/ fon(x))dx  [42].  Since  J(fi,„,  fo,n)  is  a  function  of  the  AV  filter- 
estimator  parameter  n  (number  of  auxiliary  vectors),  in  the  rest  of  this  paper 
we  will  use  the  notation  J(n)  to  represent  the  J-divergence  distance  between 
fi,„(x)  and  /o.n(x).  For  the  Gaussian  approximated  pdfs  /i,„  arid  /o,„,  we  have 
D(fi,n,fo,n)  =  D(/o,n,/i,„)  =  [2n(n)]2/2cr?+N(n)  and  the  J-divergence  simplifies 
to 


J(n)  = 


4  mV) 
CT/+v(«) 


(26) 


Expression  (26)  justifies  our  choice  of  the  output  J-divergence  as  one  of  the 
underlying  rules  for  the  selection  of  the  best  AV  filter  estimator.  We  recall  that 
under  the  same  Gaussian  approximation  of  the  conditional  filter-output  pdfs 
the  filter  output  signal-to-interference-plus-noise-ratio  (SINR)  can  be  expressed 
as  J{ri)/ 4  and,  consequently,  the  bit-error-rate  (BER)  as  Q(-JJ(n)/2),  where 
Q(x)  =  /~  1/V2tt  exp(-«2/2)dw.  To  this  extent,  maximization  of  the  output  J- 
divergence  in  (26)  implies  minimization  of  the  BER.  Therefore,  we  propose  to 
select  the  estimator  from  the  generated  sequence  of  AV  filter  estimators  that 
exhibits  maximum  estimated  J-divergence. 


(1)  Supervised  output  J-divergence  rule.  Exploiting  the  symmetry  of  /i,„(-) 
and  fo.n (■)>  we  can  show  in  a  straightforward  manner  that 

4£2{&iRe[w„*(M)r]}  2 

Var{f>i  Re[w^(M)r]} 

4[E,=±1  E{i  Re[w„ff  (M)r]|f.x  =  f }  Pr(fci  =  Q]2 
E(=±iVar{Re[w«(M)r]|h  =  i}Pr(fci=0  ’ 


where  Var{  )  and  Pr{  }  denote  variance  and  probability,  respectively.  Assuming 
availability  of  a  pilot  information  bit  sequence  {fci(m))^=1,  we  propose  to 
estimate  J(n)  by  estimating  statistical  expectations  and  probabilities  via 
sample  averaging  and  frequencies  of  occurrence,  respectively.  We  note  that 
although  (27)  and  (28)  are  ideally  equivalent  (when  all  statistical  quantities  are 
known),  this  is  not  the  case  in  general  when  estimated  measures  are  considered. 
So,  let  {p?\ =  [Re[w^(M)rm] : b\(m)  =  +1,  m  =  1,2,3..., M} 

and  [p~,p2,Pz . Pm2)  =  {Re[wf  (M)rm]:hi(w)  =  -1,  m  =  1,2,3,  be 

the  sets  of  all  filter  outputs  under  b\{m)  =  +1  and  b\(m)  =  —1,  respectively 
(Mi,  M2  #  0  and  Mi  +  M2  =  M).  First,  it  can  be  shown  in  a  straightforward 
manner  that  the  estimator  of  the  numerator  of  (28)  (that  is,  the  weighted 
average  of  the  sample  average  mean  of  the  set  [p^mLi  and  the  sample 
average  mean  of  the  set  [Pm)mti  weighted  by  the  frequency  of  each  set)  is 

equivalent  to  the  estimator  of  the  numerator  of  (27)  4 p2(n)  where  p(n)  = 
£x(m) pm/(Mi  +  M2)  (in  fact,  £(«)  is  the  minimum  variance  unbiased 
estimator  of  p(n)  [43,  p.  178]). 
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Estimators  of  the  denominator  of  (27)  and  (28)  are  examined  in  the  following 
proposition.  The  proof  is  included  in  the  Appendix. 


PROPOSITION  3.  Consider  the  estimator  of  of+N(n),  which  is  the  weighted 
average  of  the  sample  average  variance  of  the  set  ond  the  sample 

average  variance  of  the  set  [p~  weighted  by  the  frequency  of  occurrence  of 

each  set 


where 


a2 


dtin)  = 


Mi 


M\  +  M2 


i-2+ 


(»)  + 


M2 


Mi  +  M2 


d2  ( n ), 


(29) 


-  5:  -  a»>2.  a+<«> =^X>;, 


Ml 


A  1 


Mi 


Ml 


m=l 

M2 


Mi 


m=l 

M2 


m~l 


m= 1 


a7W^  am $  M 2  are  a5  defined  previously.  Consider  also  the 

direct  sample  average  estimator  of  of +N{n), 


j  Afi+M2 

^(w)  =  M1  +  M2  -  A(«)]2,  (30) 

WI— 1 

where  pm  is  given  by  (23).  The  estimators  of  (n)  and  of{ri)  exhibit  the  following 
properties:  (i)  They  are  both  biased  and  (ii)  of  (n)  exhibits  smaller  MSE  from  the 
true  value  than  of(n). 

Utilizing  Proposition  3,  estimators  for  the  filter-output  J-divergence  become 
readily  available.  The  following  theorem  identifies  their  relative  merits.  The 
proof  is  included  in  the  Appendix. 

THEOREM  2.  Define  the  two  supervised  estimators  of  the  output  J-divergence 
=  4fi2(n)/of(n)  and  JSt 2(«)  =  4 fi2(n)/of(n),  where  the  subscript  “S” 
identifies  a  supervised  implementation.  Then,  for  a  given  information  bit  pilot 
sequence  of  size  M  =  M\  +  M2,  where  M\  and  M2  are  the  cardinalities  of  the  sets 
{b\{m)  =  4-1}  and  {fei(m)  =  -1),  respectively,  both  estimators  are  biased  while  the 
MSE  of  Js,2(n)  is  less  than  the  MSE  of  Jsti(n);  i.e., 


E 


{[ 


h,2(n)  - 


4/x2(n)12] 
af+N 1 


(31) 


where  4 g2(n)/crf+N(n)  is  the  true  value  of  J (n)  as  given  by  (26). 

Using  the  preferred  estimator  Js,2(n),  the  supervised  implementation  of  the 
output  J-divergence  AV  filter  estimator  selection  rule  takes  the  final  form  given 
by  the  following  criterion. 
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CRITERION  2.  For  a  given  information  bit  pilot  sequence  of  size  M,  the 
supervised  J -divergence  AV  filter  estimator  selection  rule  chooses  the  estimator 
w „2(M)  with  n-2  auxiliary  vectors  where 

7i2  =  argmax{  Jjt2(«)} 

f  4[£  hW  Re[w„tf  (M)rm]]2  1 

argm"aX  1 1  Em= l  PiO")  Re[w„"  (M)rm]  -  A(«)]2  )  ’ 

(2)  Unsupervised  (blind)  output  J- divergence  rule .  The  blind  implementa¬ 
tion  of  the  rule  is  obtained  by  substituting  the  information  bit  b\  in  (27)  by  the 
detected  bit  b\  —  sgn[Re[w^(M)r]]  (output  of  the  sign  detector  that  follows  the 
linear  filter).  In  particular,  using  b\  in  place  of  b\  in  (27)  we  obtain  the  following 
J-divergence  expression 


4E2{fr1Re[w„tf(M)r]}  _  4E2{]Re[wntf(M)r]|} 
Var{6i  Re[w* (M) r]}  Var{|  Re[w«(M)r]|}  ’ 


(33) 


where  the  subscript  “B”  identifies  the  blind  version  of  the  J-divergence  function. 
The  following  proposition  provides  the  conditions  under  which  Jb  (n)  is  nearly 
equal  to  J(n).  The  proof  is  included  in  the  Appendix. 

PROPOSITION  4.  If  fi(n)/(ai+N(n))  »  1,  i.e.,  the  filter  output  SINR  is  signifi¬ 
cantly  higher  than  0  dB,  then  Js(n) «  J(n). 

To  estimate  Jb  (n)  from  a  data  record  of  finite  size,  we  substitute  the  statistical 
expectations  in  (33)  by  sample  averages.  The  following  criterion  summarizes  the 
corresponding  AV  filter  estimator  selection  rule. 

CRITERION  3.  For  a  given  data  record  of  size  M,  the  unsupervised  (blind)  J- 
divergence  AV  filter  estimator  selection  rule  chooses  the  estimator  w„3(A/)  with 
n%  auxiliary  vectors  where 


7i3  =  argmax{yB(7i)} 


l 


—  argmax 

n  l 


4[^Em=xlRe[wnH(M)rm]|]2 


£m=ilRe[w»(M)rm]|2  -  [£  E«=x  I  Re[w»(M)rm]|]! 


;}• 


(34) 


EMULATION  STUDIES 

We  examine  the  performance  of  the  proposed  short  data  record  AV  filter 
estimator  selection  rules  for  a  DS-CDMA  system  with  K  users,  spreading 
gain  L,  and  multipath  fading  reception  by  a  narrowband  antenna  array  with  N 
elements.  All  elements  experience  identical  fading.  Let  J  denote  the  number  of 
chip  interval  spaced  paths  per  baseband  user  signal.  After  conventional  carrier 
demodulation,  chip-matched  filtering,  and  sampling  at  the  chip  rate  over  a 
multipath  extended  symbol  interval  of  L  +  J  —  1  chips,  the  L+J-l  data  samples 


212 


Digital  Signal  Processing  V 61. 12,  Nos.  2,3,  April  and  July  2002 


from  the  i  th  antenna  element,  /  =  1, 2, . . . ,  N,  are  organized  in  the  form  of  a 
vector  given  by 

K  J 

rrn  -  y^.y'skjjEkibkjm^k,!  +  (m)s^,  +  (m)ei£  t)akj[i]  +  nj^, 

*=lr=l 

"i  =  l . M,i  =  l,...,N.  (35) 

In  (35),  with  respect  to  the  fcth  user  signal,  Ek  is  the  transmitted  energy,  bk(m), 
bk  ini),  and  (m)  are  the  present,  the  previous,  and  the  following  transmitted 
bits,  respectively,  and  ckJ  is  the  coefficient  of  the  fth  path  of  the  kth  user 
Bignal.  The  channel  coefficients  are  modeled  as  independent  zero-mean  complex 
Gaussian  random  variables  that  are  assumed  to  remain  constant  over  the  filter 
adaptation  data  record  of  size  M.  s*,,  represents  the  (7  -  l)-zero-padded  and 
(t  -  l)-right-shifted  version  of  the  signature  of  the  Jtth  user  s*;  s* ,  is  the  0- 
filled,  L -left-shifted  version  of  s*,,;  and  af  t  is  the  0-filled,  L-right-shifted  version 
of  8kJ.  Finally,  represents  additive  complex  white  Gaussian  noise  and  akJ[i] 
denotes  the  ith  coordinate  of  the  array  response  vector  a*,,  that  corresponds  to 
the  fth  path  of  the  Arth  user  signal 

ak,'[i]  =  exp|j27r(i  - 1)— j,  i  =  (36) 

where  0*,,  is  the  angle  of  arrival,  X  is  the  carrier  wavelength,  and  q  is  the  inter¬ 
element  spacing  (in  our  studies  we  set  q  =  A/2). 

(We  vectorize  the  (L  +  J  —  1)  x  N  space-time  received  data  matrix 
rm\  • .  • ,  im^l  to  form  the  joint  space-time  data  vector  rm,  which  is  a  (L  -f  J 
—  l)N-long  column  vector: 

rm  =  Vec{[r£>,  r<?>,  ....  r^](L+J_1)xN}.  (37) 

The  joint  space-time  RAKE  filter  for  user  1  is  Vi  =  Ebi[*mb\{tri)}  =  Vec{[vi  i, 
vi,2>  •  ••»  vi,/v]}  (£^{  }  denotes  statistical  expectation  with  respect  to  £>i(m)), 

and  vu  =  cht^i,taijV]f  1  =  1, 2, . . . ,  N.  The  MMSE-MVDR  filter  is  built 
with  constraint  vector  v  =  vi,  desired  response  w^vi  =  1,  and  autocorrelation 
matrix  R  =  £{rmr"}. 

We  choose  K  —  20,  N  =  5,  J  —  3  paths  with  independent  zero-mean  complex 
Gaussian  fading  coefficients  of  variance  one  (i.e.,  £{|q,,|2}  =  1)  and  Gold 
signatures  with  processing  gain  L  =  31.  The  total  SNR’s  (over  the  three  paths) 
of  the  19  interferers  are  set  at  SNR^  =  6  dB,  SNR7^  =  7  dB,  SNR^i3  =  8  dB, 
SNR14-i5  =  9  dB,  and  SNRi$_20  =  10  dB.  The  space-time  product  (filter  length) 
equals  (L  +  J  -  1)N  —  (31  +  2)5  =  165.  All  experimental  results  that  follow  are 
averages  over  100  different  channel  realizations  and  10  independent  data  record 
generations  per  channel. 

We  first  examine  the  performance  of  the  AV  filter  estimator  selection  rules 
under  the  assumption  that  no  info-bit  pilot  sequence  is  available.  The  data 
record  size  is  set  equal  to  M  =  230  while  the  total  SNR  of  the  user  of  interest 
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FIG.  9.  Histogram  of  the  two  differences  n1  —  nopt  and  —  nopt  where  n\  is  the  CV-MOV  choice 
and  n3  is  the  blind  J-divergence  choice  (M  —  230,  SNRi  =  8  dB). 


is  set  at  SNRi  =  8  dB.  In  Fig.  9,  we  plot  the  empirical  pdf  of  the  differences 
(nj  -  n0pt)  and  (/13  -  nopt)  where  n\  and  M3  denote  selections  according  to 
Criteria  1  and  3,  respectively,  while  nopt  denotes  the  “genie”  maximum  SINR 
optimum  choice  of  the  number  of  auxiliary  vectors.  We  observe  that  both  criteria 
provide  a  reliable  estimate  of  the  genie-assisted  optimum  number  of  auxiliary 
vectors. 

The  overall  short  data  record  adaptive  filter  performance  is  examined  in 
Figs.  10  and  11.  In  Fig.  10,  we  plot  the  BER5  of  the  AV  filter  estimators 
wni(M)  and  w„3(Af)  as  a  function  of  the  SNR  of  the  user  of  interest  for  data 
records  of  size  M  =  230.  The  BER  curve  of  the  genie-assisted  maximum  SINR 
optimum  filter  choice  w„opt(M)  as  well  as  the  corresponding  curves  of  the  ideal 
MMSE-MVDR  filter  wmmse/mvdr,  the  SMI  filter  estimator  Woo(M),  the  S-T 
RAKE  matched  filter  (MF)  wo(M)  =  Vi,  and  the  multistage  filter  [9, 10]  with  the 
preferred  number  of  stages6  1  =  1  are  also  included  for  comparison  purposes. 
We  observe  that  both  w„x(M)  and  w „3(M)  are  very  close  to  the  genie  optimum 
AV  filter  estimator  choice  and  outperform  significantly  the  SMI  filter  estimator, 
the  multistage  filter  estimator,  and  the  matched  filter.  We  also  observe  that  for 
moderate  to  high  SNRs  of  the  user  of  interest,  the  J-divergence  selection  rule  is 
slightly  superior  to  the  CV-MOV  selection  rule.  The  opposite  is  true  in  the  low 
SNR  range.  This  is  explained  by  the  fact  that  the  J-divergence  approximation 

6  The  BER  of  each  filter  under  consideration  is  approximated  by  Q  (VSINRom )  [41],  since  the 
computational  complexity  of  the  BER  expression  for  this  antenna  array  CDMA  system  prohibits 
exact  analytic  evaluation. 

6  In  [44],  it  is  argued  that  /  =  7  (D  =  8  in  the  notation  of  [44])  stages  are  “nearly  optimal  over  a 
wide  range  of  loads  and  SNRs.  ” 
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FIG.  10.  BER  versus  SNR  for  the  user  signal  of  interest  {M  -  230). 


180  200  220  240  260  280  300  320  340  360 

Date  Record  Size  M 

FIG.  11.  BER  versus  data  record  size  (SNRi  =  8  dB). 


J («)  Jb( n)  used  in  Proposition  4  is  less  accurate  for  low  filter  output  SINR 
values.  On  the  other  hand,  for  high  filter  output  SINRs  the  discrimination 
capability  of  the  CV-MOV  rule  is  not  as  sharp. 

Finally,  Fig.  11  repeats  the  study  of  Fig.  10  as  a  function  of  the  data  record 
size.  The  SNR  of  the  user  of  interest  is  fixed  at  8  dB. 
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6.  CONCLUDING  REMARKS 

In  this  article  we  relied  strictly  on  statistical  conditional  optimization 
principles  to  derive  an  iterative  algorithm  that  starts  from  the  white-noise 
matched  filter  and  converges  to  the  MMSE-MVDR  filter  solution  for  any  given 
positive  definite  input  autocorrelation  matrix.  The  conceptual  simplicity  of 
the  employed  conditional  optimization  criteria  led  to  a  computationally  simple 
iteration  step.  We  analyzed  basic  algorithmic  properties  and  we  established 
formal  convergence  to  the  MMSE-MVDR  filter. 

When  the  input  autocorrelation  matrix  is  substituted  by  a  sample-average 
(positive  definite)  estimate,  the  algorithm  generates  a  sequence  of  filter 
estimators  that  converges  to  the  familiar  sample  matrix  inversion  unbiased 
estimator.  The  bias  of  the  generated  estimator  sequence  decreases  rapidly  to 
zero  while  the  estimator  covariance  trace  increases  slowly  from  zero  (for  the 
initial,  fixed-valued,  matched-filter  estimator)  to  the  asymptotic  covariance 
trace  of  SMI.  Sequences  of  practical  estimators  that  offer  such  exceptional 
control  over  favorable  bias-covariance  balance  points  are  always  a  prime 
objective  in  the  estimation  theory  literature.  Indeed,  for  finite  data  record  sets, 
members  of  the  generated  sequence  of  estimators  were  seen  to  outperform  in 
MS  estimation  error  LMS  and  RLS  types,  SMI  and  diagonally  loaded  SMI, 
and  orthogonal  multistage  decomposition  filter  estimators.  In  addition,  the 
troublesome,  data-dependent  tuning  of  the  real-valued  LMS  learning  gain 
parameter,  the  RLS  initialization  parameter,  or  the  SMI  diagonal  loading 
parameter  is  replaced  by  an  integer  choice  among  the  first  several  members 
of  the  estimator  sequence.  Two  data-driven  criteria  were  proposed  for  the 
identification  of  the  best  AV  filter  estimator  in  the  sequence.  The  first 
criterion  calls  for  the  minimization  of  the  cross-validated  filter-estimator 
output  variance.  The  second  criterion  calls  for  the  maximization  of  the  J- 
divergence  of  the  filter-estimator  output-conditional  distributions.  Simulation 
studies  examined  and  compared  the  operational  characteristics  of  the  proposed 
selection  methods.  With  respect  to  the  relative  merits  of  the  minimum  cross- 
validated  output  variance  and  the  maximum  output  J-divergence  selection 
rules,  we  observed  that  for  moderate  to  high  output  SINRs  the  latter  method 
appears  superior  to  the  former  (for  high  SINRs  the  cross-validated  minimum 
output  variance  rule  is  not  as  sharp  in  discrimination  ability).  In  contrast,  in 
low  output  SINR  the  J-divergence  method  is  somewhat  lacking  in  performance 
(technically,  the  approximation  in  Proposition  4  is  less  accurate  for  near  0  dB  or 
lower  output  SINR  values).  As  a  final  general  comment,  the  use  of  a  sufficiently 
long  antenna  array  in  combination  with  an  “effective”  interference  suppressive 
filter  can  result  in  high  output  SINR  which  favors  the  J-divergence  selection 
rule,  even  when  the  transmitted  energy  of  the  user  of  interest  is  much  lower 
than  that  of  the  interferers. 

The  auxiliary-vector  algorithm  in  Fig.  2  together  with  Criteria  1,  2,  and  3 
form  a  complete  toolbox  for  state-of-the-art  estimation  of  MMSE-MVDR  filters. 
The  developments  are  of  particular  interest  in  high-dimensional  adaptive  signal 
processing  applications  that  rely  on  data  records  of  limited  size. 
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APPENDIX 


Signature  Assignment  for  the  DS-CDMA  Example  of  Sections  3  and  4 

The  matrix  832*13  =  [si  82  •••  813]  with  columns  the  signature  vectors 
81 , 82 , ....  813  is  given  below. 
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Proof  of  Proposition  3.  The  quantities  (l/Mi)£„ii(Pm  -  £+00)2  and 
a/*2>ij£i<rt;  -  £-(«))2  are  the  ML  estimators  for  the  variance  of  the 
filter  output  conditioned  on  b\  =  +1  and  b\  =  -1,  respectively  [43,  p.  179]. 
Both  estimators  are  biased.  (In  fact,  their  unbiased  counterparts  that  have 
multiplying  factors  l/(Mi  - 1)  and  1/(M2  - 1)  instead  of  1/Mi  and  1  /M2, 
respectively,  exhibit  higher  MSE.)  The  MSEs  of  the  estimators  of  interest  &?(n) 
and  <rf  (n)  are  as  follows: 


"*{[( 


Mi  4-  M2 


~M  i  +  M2  ) 


4 


'LnLliPm  -  A  (”))2 

M\  4  M2 


l2] 

M!+M2  )\  ] 
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(2M1  -  1) af+N(n)  (2 M2  -  1) <xf+N{n)  2of+N(n) 

(Mi  +  M2)2  (Mi  +  M2)2  +  (Mi  +  M2)2 

Mi  +  M2 


(A.2) 


and 


MSEaf  („)  =  Ei  (ai (")  -  °72+jv(«))  }  = 


[2(Mi  +  M2)  -  1]CT,4+W(n) 
(Mi  +  M2)2 


(A.3) 


Thus, 


MSE 


(«)  ‘ 


■  MSE-2/„,  = 


JI+N 


(n) 


a2(")  (Mx  +  M2)2 


>0. 


(A.4) 


Proof  of  Theorem  2.  A+(«)  and  o2+{n)  are  independent  random  variables 

with  distributions  fi+(n)  ~  <rf+N(n)/Mi)  and  (Mi/of+Ar(n))d2+(n)  ~ 

^2  _i  [45]  (x^x-i  denotes  the  chi-square  distribution  with  (Mi  -  1)  degrees  of 
freedom).  Similarly,  and  o2~(n)  are  independent  random  variables  with 

distributions  / l~(n )  ~  Af(— n(n),  crf+N(n)/M2)  and  (M2/cr?+N(n))e2  (n)  ~  x«2_i- 
Furthermore,  (/t+(n),  a2+(n))  and  (/x”(n),  d2-(n))  are  mutually  independent  be¬ 
cause  (/t+(n),  d'2"i"(w))  and  (/i~(rt),a2~(n))  sr©  evaluated  from  two  independent 
training  sets. 

Since  jx(n)  =  (A/i/Af)/i+(n)  —  (M2/ M)jl~ (n)  and  <J2(n)  =  (Mx/M)a2+(n)  + 
(Af2/A/)<r2“(n),  £(«)  and  of  (n)  are  independent  random  variables  with  distri¬ 
butions  £i(n)  ~  Ar(/x(n)J  <yf+N(n)/M)  and  (M/of+Jy(«))af  (n)  ~  X^-2  where  M  = 
Mi  +  M2  is  the  size  of  the  given  information  bit  pilot  sequence.  Therefore, 


£{/s,l(n)}  =  4£{A2(«)}^|T^|  =4 
4/i.2(n)  M  4 


r°rH-N^n'*  ,  2y 

—  +  fi  (n) 

M 


M 


W+w(n)(M-4) 


M  — 4’ 


(A.5) 


E{7|i(n)}  =  16£{A4(«)}£{t^}  =  16 


m4(«)  + 


6^2  (n)of+N(n)  3of+N(n) 


M 


M2 


M2 


crf+N(n)(M-4)(M-6) 


(A.6) 


The  inverse  moments  £{l/of  (n)}  and  £{l/d4(n)}  exist  for  M  >  4  and  M  >  6, 
respectively,  and  are  given  by  [46] 


E 

E 


f  1  1 

M 

(A.  7) 

l  of  (n)  J 

of+Jv(n)(M-4)’ 

(  1  1 

M2 

(A.8) 

l  o4(n)  J 

^  ~  (rf+N(n)(M  —  4)(M  —  6) 
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Thus,  the  MSE  of  Js,i(n)  is 


MSB,  4  e{  [/„<„)  -  4^12}  =  — - 

v  L  cr/ + iy  (w )  J  J  (M 


16  16  16//4  (n) 

-4)(M-6)a  M-4b+ af+N(n)’ 


where 


A  [,t4(n)  ^{n)o}+N{n)  3a4+jV(n)-|  A/2 

L  M  +  A/2  J  af+N(n) 


±\°f+N{n)  2  12V(«) 

L  M  J  °,+N(n) 


(A.  10) 


On  the  other  hand,  (i(n)  and  ohn)  are  independent  random  variables 
with  distributions  £(n)  ~  M(ii(n),af+N(n)/M)  and  (M/a?+N(n))&$(n)  ~  xl_v 
respectively.  Therefore, 

£{i5,2(«)}  =  4  [ +  m2(J  - - - 

L  M  J  crf+N(n)(M  -  3) 


4m2  (w)  M  (  4 

*/+*(«)  M~3  +  M-3’ 


^{422(«)}  =  16  M4(«)  + 


,4/-„s  ■  ^2Wcrf+N(n)  Sof+N(n) 


(A.  11) 


°?+N(n)(M  -3)(M  -5) 


Thus,  the  MSE  of  /5  2(«)  is 


MSE24£|  js2(n ) 


4/z2(«)  j2’ 


16  _  16  16m4(h) 

(M-3)(A/-5)  M-3  **  („)’ 


(A.  12) 


(A.  13) 


where  the  first  and  second  inverse  moments  £{l/a|(n)}  and  E{l/a£(n)}  exist  for 
M  >3  and  M  >  5,  respectively,  and  a  and  b  are  given  by  (A.  10). 

From  (A.9)  and  (A.13),  assuming  M  >  6  which  is  the  condition  for  all  inverse 
moments  to  exist,  we  obtain 

MSEi  -MSE2  = - — - _ +  -  -  b- 

(Af  —  4)(Af  —  6)  (A/-3)(A/-5)  A/-3  M- 4 

16  2A/-9 

(M  -  3) (A/  -  4)  [(M  -  5) (A/  -  6)°  ”  _ 

16  [  2(M  -  5)  .1 


>  (M-3)(A/-4)  [(A/-5)(A/-6)a 

_  32[6A/m4(w)  +  5A/M2(n)or;2+Af(fi)  +  6 nHn)<rf+ff(n)  +  3o4+w(n)] 
(M  -  3)  (A/  -  4)  (A/  -  6)0-^  (n) 

>  0.  (A.  14) 
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Proof  of  Proposition  4.  Define  Y  =  |  Re[w^(M)r]|.  The  pdf  of  Y  is 


fr(y)  = 


—(y  —  m(m))2  ,  „  „-0’  +  /iW)z 

exp  9 — — 1-  exp  - 


V^7TCT/+Ar(n)  L  2 a?+N(n)  *  2 o'f+N{n)  J 

where  U (v)  is  the  unit  step  function.  The  mean  of  Y  is 


U(y),  (A.  15) 


E{Y } 


=1 


yfr(y)  dy  =  n(n)  +  2or/^ffl  exp 


\Z2tz 

-2(,<n,e(^o)' 


(  m2(«)\ 

V  2a f+N) 


(A.  16) 


The  series  expansion  of  Q(x)  is  [47] 

,2n 


1  f  x2\  (  1  1-3 

ew  =  ^exp(“Tj{1-^  +  7r  +  ' 


< — 1)”1  •  3  •  (2n  —  1) 


r2  n 


| 


(A.  17) 


where  R„  =  (-l)n+1l  •  3  -(2n  +  1) XT’  ^t2n+i  exp(-f2/2) dt  is  the  remainder 
which  is  always  less  (in  absolute  value)  than  the  first  neglected  term.  If 
li(n)/<Ji+N(n) »  1,  (A.16)  can  be  written  as 


E{Y } 


=  /i(n)|l  + 


2a/+Af(») 

■s/27r/i(«) 


exp 


/  M2(»)  \ 

\  2 crf+N(n)J 


_w.>  ^w_')r_0r(-_!i(^y2i]i 

V2jr n(n)  \  2<y  f+N(n)J  L  \(ri+N(n)J  J  J  J 


/  sfi  ,  2  M2(")  \r>\f  V(n)  \  3]\ 

_/"(n)l  2af+N(n)J  [\crI+N(n))  J| 


The  variance  of  Y  is 

VarJF}  =  E{Y2}  -  E2{Y]  =  »2(n) +of+N(n)  -  E2{Y) 
and  using  (A.  18)  we  may  approximate 

VarJF}  %  crf+N(n). 

Finally,  from  (A.  18)  and  (A.20)  we  obtain 

4  E2{Y]  4fi2(n) 


(A.  18) 


(A.  19) 


(A.20) 


JB(n)  = 


VarJF} 


JI+N 


(n) 


=  /(«). 


(A.21) 
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Data-record-based  Criteria  for  the  Selection 
of  an  Auxiliary- Vector  Estimator 
of  the  MMSE/MVDR  Filter 

Haoli  Qian,  Student  Member,  IEEE,  and  Stella  N.  Batalama,  Member,  IEEE 


Abstract — When  the  auxiliary- vector  (AV)  filter  genera¬ 
tion  algorithm  utilizes  sample  average  estimated  input  data 
statistics,  it  provides  a  sequence  of  estimates  of  the  ideal 
minimum-mean-square-error  (MMSE)  or  minimum-variance- 
distortionless-response  (MVDR)  filter  for  the  given  signal  process¬ 
ing/receiver  design  application.  Evidently,  early  non-asymptotic 
elements  of  the  sequence  offer  favorable  bias/variance  balance 
characteristics  and  outperform  in  mean-square  filter  estimation 
error  the  unbiased  sample-matrix-inversion  (SMI)  estimator  as 
well  as  the  (constraint)-LMS,  RLS,  ‘‘multistage  nested  Wiener 
filter,”  and  diagonally-loaded  (DL)-SMI  filter  estimators.  Select¬ 
ing  the  most  successful  (in  some  appropriate  sense)  AV-filter 
estimator  in  the  sequence  for  a  given  data  record  is  a  critical 
problem  that  has  not  been  addressed  so  far.  In  this  paper 
we  deal  exactly  with  this  problem  and  we  propose  two  data 
driven  selection  criteria.  The  first  criterion  minimizes  the  cross- 
validated  sample  average  variance  of  the  AV-filter  output  and 
can  be  applied  to  general  filter  estimation  problems;  the  second 
criterion  maximizes  the  estimated  J-divergence  of  the  AV-filter- 
output  conditional  distributions  and  is  tailored  to  binary  phase- 
shift-keying  (PSK)-type  detection  problems. 

Index  Terms — Adaptive  filters,  antenna  arrays,  code  divi¬ 
sion  multiaccess,  cross-validation,  interference  suppression,  J- 
divergence,  low  sample  support,  short-data-record  estimation. 

I.  Introduction  and  Background 

N  this  paper,  we  deal  with  the  broad  and  challenging 
problem  of  designing  adaptive  receivers  from  a  limited  data 
record  of  size  commensurate  to  the  channel  coherence  time. 
The  coherence  time  of  the  channel,  which  is  reciprocal  to  the 
channel  rate-of-change,  will  determine  the  size  of  the  data 
record  (block)  available  for  receiver  adaptation  and  redesign 
(the  shorter  the  coherence  time  of  the  channel,  the  smaller  the 
available  data  record).  For  presentation  purposes  we  consider 
a  simple  block  fading  channel  model  that  assumes  fixed 
fading  over  the  symbols  in  the  block  and  independent  fading 
across  blocks.  After  receiver  adaptation  based  on  a  specific 
data  record  (block)  is  completed,  we  utilize  the  obtained 
receiver/filter  estimate  to  detect  the  information  bits  contained 
in  that  same  block.  Our  study  focuses  on  the  auxiliary- vector 
(AV)  algorithm  [l]-[3]  that  generates  a  sequence  of  linear 
filters/receivers  that  we  call  AV-filters/receivers.  Members  of 

This  work  was  supported  by  the  National  Science  Foundation  under  Grant 
ECS-0073660  and  Grant  CCR-0 129903  and  by  the  Air  Force  Office  of 
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in  part  at  the  Thirty-Fourth  Asilomar  Conference  on  Signals,  Systems  and 
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haoqian@eng.buffalo.edu;  batalama@eng.buffalo.edu). 


this  sequence  -when  evaluated  (estimated)  from  a  data  record 
of  limited  size-  were  seen  to  exhibit  superior  short-data-record 
adaptive  filtering  performance.  Our  objective  in  this  paper  is 
to  develop  data  driven  criteria  for  the  selection  of  the  most 
successful  AV-filter  estimator  in  the  sequence  for  a  given 
record  of  input  observations  of  size  commensurate  to  the  rate- 
of-change  of  the  receiver  operational  environment. 

The  auxiliary-vector  (AV)  filter  generation  algorithm  given 
in  [1]— [3]  is  a  statistical  optimization  procedure  that  produces 
a  sequence  of  linear  filters  (AV-filters)  that  converges  to 
the  MMSE/MVDR  solution  under  an  ideal  setup  (perfectly 
known  input  data  autocovariance  matrix).  The  developments 
in  [l]-[3]  can  be  viewed  as  an  extension  of  the  work  in 
[4]-[8]  to  non-orthogonal  multiple  AV  MMSE/MVDR  filter 
synthesis.  The  AV  filter  generation  algorithm  exhibits  the 
following  key  attributes:  (i)  Utilizes  non-orthogonal  auxiliary 
vectors  [1]— [31;  (ii)  the  auxiliary  vectors  are  selected  according 
to  the  maximum  cross-correlation  (MCC)  criterion  [5];  (Hi) 
the  filter  sequence  is  created  based  on  statistical  conditional 
optimization  [6]-[8];  and  (iv)  each  (non-asymptotic)  filter  in 
the  sequence  has  a  form  that  does  not  involve  any  explicit  or 
implicit  matrix  inversion  operation  (in  contrast  for  example  to 
reduced-rank  filtering  methods  such  as  in  [9], [10]). 

The  AV  algorithm  is  summarized  in  Fig.  1.  In  terms  of 
notation,  w  €  CL  represents  a  complex  L-tap  linear  filter 
and  R  =  E{ rrH}  is  the  input  data  autocorrelation  matrix 
(E  {*}  denotes  statistical  expectation,  H  denotes  the  Hermitian 
operation,  and  r  €  CL  is  the  random,  zero  mean  without 
loss  of  generality,  complex  input  vector  of  dimension  L );  v 
is  the  input  signal  vector  direction  to  be  protected  during 
filtering  (for  general  MMSE  filters  v  is  set  equal  to  the 
statistical  cross-correlation  between  the  desired  scalar  filter 
output  b  e  C  and  the  input  vector  r,  v  =  E{rb*}).  For 
presentation  purposes,  in  Fig.  1  we  assume  that  the  filtering 
operation  is  supposed  to  be  “distortionless”  in  the  v  vector 
direction,  i.e.  wHv  =  1  (the  developments  can  be  generalized 
in  a  straightforward  manner  to  cover  filter  constraints  of  the 
form  wHv  =  p  for  any  complex  scalar  p  [2]).  The  vectors 
gd,  d  =  1, 2, . . . ,  are  referred  to  as  “auxiliary  vectors  ”  The 
auxiliary  vector  g<*  is  required  to  be  orthogonal  only  to  the 
constraint  vector  v  itself  and  is  not  necessarily  orthogonal 
to  the  earlier  auxiliary  vectors  g*,z  =  1,2, ...,d  —  1;  the 
auxiliary  vector  g<*  is  chosen  to  maximize  the  magnitude  of  the 
statistical  cross-correlation  between  the  previous  filter  output 
and  the  projection  of  the  input  data  onto  the  auxiliary  vector 
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itself,  i.e.  gd  =  argmax  |w^_jRg|  subject  to  gjv  =  0 

and  Ed  Ed  =  1.  The  corresponding  scalar  AV  weight  fid 
is  chosen  to  minimize  the  new  filter  output  variance,  i.e. 
Hd  =  arg  min  wJRwj.1  We  emphasize  that  by  relaxing  the 
orthogonality  condition  among  auxiliary  vectors,  the  length  of 
the  generated  sequence  of  filters  is  not  limited  by  the  vector 
space  dimension  as  in  the  case  where  orthogonality  between 
auxiliary  vectors  is  maintained  [6]-[8],  The  AV  algorithm  in 
Fig.  1  generates  a  sequence  of  filters  wd  =  w0  -  ^igi> 
d  1,2,. ...  Formal  theoretical  analysis  of  the  sequence  of 
auxiliary-vector  filters  was  pursued  in  [2], 

For  a  given  constraint  (or  cross-correlation)  vector  v  and 
perfectly  known  input  autocorrelation  matrix  R,  the  sequence 
of  ideal  AV  filters  w0, Wi, w2, . . .  converges  to  the  ideal 
MVDR  (MMSE)  optimum  filter 


balance  and  outperform  in  mean-square  filter  estimation  error 
W smi(N),  (constraint)-LMS,  RLS,  multistage  nested  Wiener 
filter  [9],  [  10],  and  diagonally-loaded  (DL)-SMI  estimators.  In 
the  context  of  digital  wireless  communication  receivers,  the 
latter  translates  to  superior  bit-error-rate  performance  under 
short  data  record  receiver  adaptation. 

We  conclude  this  introductory  section  and  we  motivate  the 
developments  that  follow  with  an  illustration  of  the  signifi¬ 
cance  of  the  AV  parameter  d  (that  is  the  exact  number  of 
auxiliary  vectors  used  in  building  the  AV-filter  estimator)  and 
the  dependence  of  the  proper  choice  of  d  on  the  true  signal 
model,  the  constraint  vector  direction,  the  data  record  size, 
as  well  as  the  specific  data  record  realization.  We  draw  a 
signal  model  example  from  the  direct-sequence  code-division- 
multiple-access  (DS/CDMA)  literature  and  we  consider  a 


W MMSE/MVD*  -  ~WR_ly.  (1) 

We  recall  that  the  filter  in  (1)  minimizes  wwRw  and  simul¬ 
taneously  satisfies  w*v  =  1.  Not  withstanding  the  satisfying 
convergence  properties  of  the  ideal  AV  filter  sequence,  the 
main  motivation  for  the  development  of  the  AV  filtering 
framework  is  improved  adaptive  receiver  performance  when 
receiver  adaptation  is  based  on  a  short  record  of  N  input 
data.  When  the  input  autocorrelation  matrix  R  is  estimated  by 
averaging  the  outer  products  of  N  received  vectors,  R (N)  = 
n  Sn=i  ’  ffi®  MMSE/MVDR  filter  estimator  obtained 
by  using  the  sample-average  estimate  R (N)  in  place  of  R 
in  (1)  is  known  as  the  sample-matrix-inversion  (SMI)  filter 


w smi(N)  -  -^71(/Vv)v-  The  filter  w SM,(N)  exhibits  all  the 
disadvantages  associated  with  the  inversion  of  a  matrix  that 
is  estimated  from  a  finite-size  data  record  (for  example,  the 
inverse  exists  w.p.  1  only  if  the  size  of  the  data  record  N  is 
greater  than  or  equal  to  L)  and  data  record  sizes  many  times 
the  filter  length  L  are  necessary  for  w SMl(N)  to  approach 
reasonably  well  the  performance  characteristics  of  the  ideal 
wmmsb/mvdr  filter.  Unfortunately,  in  many  communications 
systems  with  typical  input  data  dimension  values  and  data 
transmission  rates,  the  data  record  (block)  size  (that  fits  within 
a  channel  coherence  period)  is  relatively  small  for  wSM/(AT) 
to  perform  well.  In  contrast  to  wSM,(AT)  and  other  potential 
reduced-rank  filter  estimators  such  as  the  “multistage  nested 
Wiener  filters”  in  [9],  (10],  the  AV-filter^estimators  obtained 
by  utilizing  the  sample-average  estimate  R(iV)  in  place  of  R 
in  Fig.  1  are  not  the  result  of  any  form  of  direct  or  indirect 
matrix  inversion  operation.  As  illustrated  in  [2],  for  a  fixed 
finite  data  record  of  size  N,  the  AV-filter  sequence  {wd(JV)}d 
based  on  the  AT-point  autocorrelation  matrix  estimate  R(JV) 
can  be  viewed  not  only  as  a  sequence  of  estimators  of  the  cor¬ 
responding  ideal  AV  filters  but  also  as  a  sequence  of  estimators 
of  the  ideal  MMSE/MVDR  filter  that  has  varying  bias  versus 
variance  characteristics  and  converges  to  wSM,(N)  (if  the 
latter  exists).  For  small  N,  the  early  non-asymptotic  elements 
of  the  sequence  of  AV  estimators  offer  favorable  bias/variance 

’Since  fidg d  is  independent  of  the  norm  of  gd,  then  so  is  wd.  Based  on 
this  observation,  in  Fig.  1  we  dropped  the  unnecessary  normalization  of  the 
auxiliary  vectors. 


system  with  K  —  20  users  with  Gold  signatures  of  length 
T  =  31  and  multipath  fading  reception  by  a  narrowband  linear 
antenna  array  with  M  =  5  elements  (all  elements  are  assumed 
to  experience  identical  fading).  We  assume  that  each  signal 
experiences  J  =  3  chip-interval-spaced  paths.  The  path  fading 
coefficients  are  modeled  as  independent  identically  distributed 
(i.i.d.)  zero-mean  complex  Gaussian  random  variables.  The 
directions  of  arrival  for  all  user  paths  are  i.i.d.  uniformly 
distributed  in  [-§,§].  We  pick  a  “user  of  interest,”  say 
user  0,  and  we  fix  the  total  SNR’s  (over  the  three  paths) 
of  the  19  interferers  at  SNR^s  =  6dB,  SNR6_7  =  7dB, 
SNR8_12  =  8dB,  SNR13_14  =  9dB,  SNR15_19  =  lOdB. 
The  space-time  product  for  this  communication  system  equals 
+  J~  1)M  =  165.  After  conventional  carrier  demodulation 
and  chip-matched  filtering  and  sampling  at  the  chip  rate  over 
a  multipath  extended  symbol  interval  of  T  +  J  -  1  chips,  the 
(T  +  J  -  1)  x  M  data  samples  (T  +  J  -  1  data  samples 
from  each  antenna  element)  are  organized  in  the  form  of  an 
L  =  165-long  joint  space-time  received  data  vector  r.  The 
input  vector  direction  to  be  protected  during  MMSE/MVDR 
space-time  filtering  is  the  effective  space-time  signature  of 
user  0  or  in  other  words  the  cross-correlation  between  the 
input  data  vector  r  and  the  desired  user  information  bit  i>0, 
Vo  =  Ebo{rb*0}. 

With  this  setup,  we  conduct  the  following  experiment.  We 
generate  an  input  data  record  of  size  N  and  we  estimate  the 
elements  of  the  AV-filter  sequence.  Then,  for  each  AV-filter  es¬ 
timate  we  evaluate  the  output  SINR  using  the  ideal  (true)  input 
statistics  and  we  record  the  "genie"  assisted  optimum  choice 
of  the  number  d  that  corresponds  to  the  AV-filter  estimate  in 
the  sequence  with  the  maximum  SINR  value.  We  emphasize 
that  the  optimum  number  d  is  a  random  variable  rather  than  a 
deterministic  function  of  N ;  for  a  given  data  record  realization 
of  size  N,  the  genie  assisted  choice  of  d  optimizes  (in 
the  maximum  output  SINR  sense)  the  filter  estimate  that  is 
evaluated  based  on  this  specific  data  record  realization.  We 
recall  that  in  practice  the  data  record  size  N  is  controlled 
by  system  specifics  and  the  coherence  time  of  the  channel 
(for  example  with  CDMA  chip  rates  at  1.25MHz,  processing 
gain  32,  earner  frequency  900MHz,  and  vehicle  speed  45mph, 
the  channel  coherence  time  is  approximately  7ms  [11]  and 
the  fading  channel  fluctuates  decisively  at  least  every  275 
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data  symbols  [7]).  We  repeat  our  experiment  1,000  times 
(100  different  channel  realizations  and  10  independent  data 
record  generations  per  channel)  and  we  show  the  empirical 
probability  density  function  of  the  random  variable  d  in  Fig.  2. 
In  Fig.  2(a),  the  data  record  size  is  set  equal  to  N  =  230  while 
the  SNR  of  the  user  of  interest  is  set  at  SNRq  =  SdB.  Figs. 
2(b)  and  2(c)  show  an  example  of  how  this  empirical  density 
changes  when  we  increase  the  sample  support  (N  =  363)  or 
decrease  the  user  SNR  ( SNRo  =  4 dB),  respectively. 

While  it  is  clear  that  the  above  “genie”  assisted  procedure 
for  the  selection  of  d  cannot  be  used  in  practice  (due  to 
the  unknown  ideal  input  statistics),  these  studies  illustrate  the 
importance  of  identifying  a  “good”  d  in  some  appropriate 
“goodness”  sense.  Accomplishing  such  a  task  is  equivalent 
to  developing  a  data-driven,  fully  adaptive  AV  filter  receiver 
that  adapts  even  its  own  structure  (i.e.  number  of  auxiliary 
vectors  used)  for  any  given  input  data  record. 

Section  II  deals  exactly  with  this  problem,  that  is  the 
selection  of  the  best  (in  some  appropriate  sense)  AV  filter  from 
the  sequence  of  AV  filter  estimates  based  on  a  given  finite 
data  record.  Two  data  driven  selection  criteria  are  proposed 
and  analyzed.  The  first  selection  rule  is  motivated  by  the 
asymptotic  minimum  output  variance  property  of  the  AV  filters 
and  can  be  applied  to  general  filter  estimation  problems. 
The  second  rule  is  related  to  the  objective  of  achieving 
maximum  stochastic  distance  between  the  AV  filter  output 
conditional  distributions  and  is  tailored  to  applications  that  can 
be  formulated  as  antipodal  hypothesis  testing  problems  (for 
example,  detection  of  PSK-type  signals  processed  by  arbitrary 
channels  and  corrupted  by  AWGN).  In  particular,  for  a  given 
data  record,  the  first  criterion  selects  the  AV-filter  estimator  (or, 
equivalently,  the  number  of  auxiliary  vectors  used  in  forming 
the  AV  estimator)  that  minimizes  the  cross-validated  average 
filter  output  variance  (energy).  The  second  criterion  selects  the 
number  of  auxiliary  vectors  that  maximizes  the  estimated  J- 
divergence  of  the  filter  output  conditional  distributions.  Both 
supervised  and  unsupervised  (blind)  implementations  of  the 
J-divergence  criterion  are  pursued  and  analyzed.  Simulation 
studies  and  comparisons  are  presented  in  Section  III  and  some 
conclusions  are  drawn  in  Section  IV. 

II.  AV-Filter  Selection  Rules 
A.  Cross-Validated  Minimum-Output-Variance  Rule  (CV- 
MOV) 

The  CV-MOV  rule  is  motivated  by  the  fact  that  minimiza¬ 
tion  of  the  output  variance  of  filters  that  are  constrained  to  be 
distortionless  in  the  vector  direction  of  a  signal  of  interest 
is  equivalent  to  maximization  of  the  output  SINR.  Cross- 
validation  is  a  well-known  statistical  method  [12],  It  is  used 
here  to  select  the  filter  parameter  of  interest  (number  of  AV’s 
d)  that  minimizes  the  output  variance  which  is  estimated  based 
on  observations  (training  data)  that  have  not  been  used  in 
the  process  of  building  the  filter  itself.  A  particular  case  of 
this  general  method  used  in  this  paper  is  the  “leave-one-out” 
method.  The  following  criterion  outlines  the  CV-MOV  AV- 
filter  selection  process. 

Criterion  1:  For  a  given  data  record  of  size  N ,  the  cross- 
validated  minimum-output-variance  AV-filter  selection  rule 


chooses  the  AV-filter  estimator  that  minimizes  the  cross- 
validated  sample  average  output  variance,  i.e. 

di  =  argnun  j  £  ™d(N\i)riri ™d(N\i)  j  (2) 

where  the  subscript  {N\i)  identifies  the  AV-filter  estimator 
that  is  evaluated  from  the  available  data  record  after  removing 
the  i-th  sample.  ■ 

It  is  straightforward  to  show  that  the  cross-validated  output 
variance  estimate  of  the  AV-filter  estimator  that  utilizes  d 
auxiliary  vectors  converges  to  the  output  variance  of  the 
corresponding  ideal  AV-filter  w d  as  the  data  record  size 

increases,  i.e.  jj  J2iLi  *'d(N\i)riri™d(N\ o  N~^  wJRwrf. 
In  addition,  it  may  be  important  to  emphasize  the  need  for  in¬ 
voking  the  cross-validation  technique  for  the  evaluation  of  the 
sample-average  output  variance.  If  sample-average  evaluation 
using  all  data  were  attempted,  then  the  selection  rule  would 

take  the  form  arg  min  wJRwj,  where  R  =  Yln=i  r"rn 
d  ^ 

and  w d  is  obtained  by  substituting  R  in  place  of  R  in  Fig.  1. 
Such  minimization,  however,  would  result  in  d  —  oo  since  we 
know  that  wJRw wfM7RwSM/  and  wSM^  is  the  filter 
that  has  minimum  sample-output- variance  WsMJRwSM/  (but, 
of  course,  the  true  output  variance  of  wSMlt  wfMJRwSM/,  is 
not  minimum). 

B.  Output  J-divergence  Rule 

While  the  CV-MOV  criterion  presented  in  the  previous 
section  can  be  applied  to  general  filter  estimation  problems, 
in  this  section  we  present  and  analyze  an  alternative  criterion 
that  is  tailored  to  applications  that  can  be  formulated  as  binary 
hypothesis  testing  problems  on  AV-filtered  data. 

We  recall  that  for  any  scalar  binary  hypothesis  testing 
problem  characterized  by  the  detector  input  conditional  distri¬ 
butions  fo  and  fi  under  hypothesis  H0  and  Hi,  respectively, 
the  J-divergence  distance  between  /o  and  fi  is  defined  as  the 
sum  of  the  Kullback-Leibler  (K-L)  distances  between  /0  and 
fi  [13] 

J(/o,/i)  =  I>(/i,/o)  +  f?(/o,/i)  (3) 

where  the  K-L  distance  of  fi  from  /0  is  defined  as 

D(fijo)  =  /^/lWlog^^dx. 

Our  choice  of  the  output  J-aivergence  as  one  of  the  un¬ 
derlying  rules  for  the  selection  of  the  AV  filter  is  motivated 
by  the  fact  that  the  probability  of  error  of  the  optimum 
Bayesian  detector  for  any  binary  hypothesis  testing  problem 
is  lower  bounded  by  a  monotonically  decreasing  function  of 
the  J-divergence  between  the  conditional  distributions  of  the 
detector  input  [13].  For  scalar  input,  the  lower  bound  is  given 
by 

Pc  >  7r07riexp{-J(/0,/i)/2}  (4) 

where  7ro,  7Ti  are  the  a  priori  probabilities  of  Ho  and  Hi, 
respectively.  Thus,  the  larger  the  J-divergence  the  smaller  the 
lower  bound  on  the  probability  of  error.  When  the  conditional 
distributions  under  Ho  and  Hi  are  Gaussian  with  the  same 
variance,  the  probability  of  error  is  an  exact  monotonically 
decreasing  function  of  the  J-divergence.  In  particular,  let 
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the  conditional  distributions  of  the  detector  input  be  f0  ~ 
■Af  [mo><72]  and  fi  ~  Jf  [/ii,a2]  where  p0  and  pi  are  the 
conditional  means  under  hypothesis  H0  and  Hu  respectively, 
and  <r2  is  the  conditional  variance  under  either  hypothesis. 
Then, 

J(fo,fi)  =  (Mi  ~Po)2/cr2  (5) 

and  the  probability  of  error  of  the  optimum  detector  is  equal 
to  _ 

pe  =  nio(  VESA  +  jp  (^iAo) 

\  2 _ y/JihJT)) 

l)  (6) 

V  2 

where  Q(x )  =  /J°  exp  ^du .  We  can  show  that  the 
right  hand  side  of  (6)  is  a  monotonically  decreasing  function  of 
J(fo,  fi)  for  any  it o  and  tti.  Thus,  the  larger  the  J-divergence 
the  easier  the  detection  problem  or,  equivalently,  maximization 
of  the  J-divergence  implies  minimization  of  the  probability  of 
error.  Due  to  the  above  properties  and  its  relationship  with  the 
probability  of  error  of  the  optimum  detector,  J-divergence  has 
been  extensively  used  in  the  detection  literature  as  a  hypothesis 
discriminant  function. 

In  the  AV  filtering  context,  we  denote  the  conditional 
distributions  of  the  AV  scalar  filter  output  under  H0  and  Hi  by 
fo,d{')  and  /i,d(-)  where  the  index  d  indicates  the  dependence 
of  the  distributions  on  the  AV  filter  parameter  d.  Then,  the  J- 
divergence  between  /0,d(-)  and  /M(  )  is  also  a  function  of 
d  and  in  the  rest  of  the  paper  will  be  denoted  as  J(d).  The 
following  criterion  outlines  the  J-divergence  AV-filter  selection 
process. 

Criterion  2:  For  a  given  data  record  of  size  N,  the  J- 
divergence  AV-filter  selection  rule  chooses  the  AV-filter  es¬ 
timator  that  maximizes  the  estimated  J-divergence  J(d) 
between  the  AV-filter  output  conditional  distributions,  i.e. 

dj  =  argmax  |j(d)| .  (7) 

■ 

To  reach  more  specific  solutions,  we  consider  scalar  binary 
hypothesis  testing  problems  that  result  from  AV-filtered  ver¬ 
sions  of  binary  antipodal  signals.  Along  these  lines,  let  the 
received  signal  vector  be  of  the  general  form 

r  =  bVEv  +  n  (8) 

where  v  e  CL  is  a  known  deterministic  signal  vector, 
E  >  0  represents  the  unknown  energy  scalar,  n  €  CL  is 
a  comprehensive  zero-mean  disturbance  vector  (i.e.,  it  may 
incorporate  inter-symbol-interference  (ISI),  multiple-access- 
interference  (MAI)  and  additive  noise  effects),  and  6  is  +1 
or  -1  with  equal  probability.  Then,  the  decision  on  H0  ( b  = 
-1)  or  Hi  ( b  =  +1)  is  based  on  the  real  part  of  the  AV 
filter  output 

p  =  Re  [w£r]  (9) 

where  is  the  AV-filter  estimator  that  utilizes  d  auxiliary 
vectors.  To  the  extent  that  the  conditional  distributions  of 
the  filter  output  p  under  H0  and  Hi  can  be  approximated 


by  Gaussian  distributions,  we  have  fo,d  ~  M  [~p(d),c2(d)] 
and  f\,d  ~  N  [p(d),o2(d)],  where  -p(d)  and  p{d)  are  the 
conditional  means  under  hypothesis  Ho  and  Hi,  respectively, 
and  o2(d)  is  the  conditional  variance  under  either  hypoth¬ 
esis.  This  Gaussian  approximation  of  the  actual  conditional 
distributions  of  the  filter  output  can  be  considered  safe  for 
“effective”  interference  suppressive  filters,  as  argued,  for  ex¬ 
ample,  in  [14]  for  the  MMSE/MVDR  linear  filter  operating 
on  CDMA  signals.  Such  a  case  study  will  be  considered  later 
in  Section  in  where  the  effect  of  the  above  approximation 
on  the  performance  of  the  output  J-divergence  selection  rule 
will  be  examined  through  comparisons  with  the  optimum 
“genie”  assisted  selection  rule.  In  the  rest  of  this  section  we 
develop  a  supervised  and  a  blind  implementation  of  Criterion 
2  for  the  selection  of  the  best  AV-filter  in  binary  hypothesis 
testing  problems  that  result  from  AV-filtered  versions  of  binary 
antipodal  signals. 

1)  Supervised  Output  J-divergence  Rule 
Exploiting  the  symmetry  of  the  conditional  densities  of  p, 
fo,d(-)  and  fitd(:),  we  can  show  in  a  straightforward  manner 
that 


J(d)  = 


4 E2  {bRe  [w»r]} 
Var  { bRe  [w^  r]} 


2 


4 


3 


E  ^  {6J2e  [w^r]  |6  =  j}  i=»r(6  =  j)  I 
j'=+*,-i _  j 

_E  Var  {bRe  [w"r]  |6  =  j]  Pr(b  —  j) 


(11) 


where  Var(-)  and  Pr(-)  denote  variance  and  probability, 
respectively.  Assuming  availability  of  a  pilot  bit  sequence 
{Hi)}iLv  our  objective  is  to  estimate  J(d).  We  propose  to 
achieve  this  by  estimating  statistical  expectations  and  proba¬ 
bilities  via  sample  averaging  and  frequencies  of  occurrence, 
respectively.  We  note  that  although  (10)  is  equivalent  to  (11) 
ideally  (when  all  statistical  quantities  are  known),  this  is  not  in 
general  the  case  when  estimated  measures  are  considered.  So, 
let  {pi,  ■  ■  •  ,p^}={Jte  [wgn] :  b(i)  =  -1,  *  =  1,  •  •  •  ,N} 

and  {p+ ,  ■  ■  ■  ,p%1}={Re  [wjr4]  :  b(i)  =  +1,  i  =  1, . . .  ,  N} 
be  the  sets  of  all  filter  outputs  under  hypotheses  H0  and  Hi, 
respectively  (NUN2  ^  0  and  Ni  +  N2  =  N).  First,  we  can 
show  in  a  straightforward  manner  that  the  estimator  of  the 
numerator  of  (1 1)  (that  is,  the  weighted  average  of  the  sample 
average  mean  of  the  set  {p~ }  and  the  sample  average  mean 
of  the  set  {pt }  weighted  by  the  frequency  of  each  set)  is 
equivalent  to  the  estimator  of  the  numerator  of  (10)  4u2(d) 

where  p(d)  =  ^<=^1+Ar2’  —  (in  fact,  p(d)  is  the  minimum 
variance  unbiased  estimator  of  p(d)  [15,  p.  178]).  Estimators 
of  the  denominator  of  (10)  and  (11)  are  examined  in  the 
following  proposition.  The  proof  is  included  in  the  Appendix. 

Proposition  1:  Consider  the  estimator  of  c2(d)  which  is 
the  weighted  average  of  the  sample  average  variance  of  the 
set  {pf}  and  *he  sample  average  variance  of  the  set  {p~}, 
weighted  by  the  frequency  of  each  set: 


d\{d)  = 


Ni 

nx  +  n2 


o1  (d)  + 


N2 

Ni  +  N2 


o2+(d) 


(12) 
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where 


52  ^  Xj  fp*  -v  w]2’ 

1  N 2 

5!+(d)  =  ^Eb/-?+W]2. 

i= 1 


Ni 


i— 1 

N2 


J  =  1 


and  {pi  },  {p+},  iVj  and  iV2  are  as  defined  previously. 
Consider  also  the  direct  sample  average  estimator  of  a2(d): 


cl(d)  = 


1 


N!  +  N2 


JVi+iV2 

£  -  £(<*)]2  (13) 


*=1 


follows  the  linear  filter).  In  particular,  using  b  in  place  of  b  in 
(10)  we  obtain  the  following  J-divergence  expression: 


JB(d)  = 


4 E2  {bite  [w*r]  | 
Var  (bRe  [wjr]  j 


4E2{|fle[wfr]|} 
Var  {|-Re  [w"rj|} 


where  the  subscript  “B”  identifies  the  blind  version  of  the 
J-divergence  function.  The  following  proposition  identifies 
conditions  under  which  Js{d)  is  nearly  equal  to  J(d).  The 
proof  is  included  in  the  Appendix. 

Proposition  2:  If  »  1,  i.e.  the  filter  output  SINR 
is  significantly  higher  than  OdB,  then  Js(d)  «  J(d).  ■ 


where  p<  =  Re  [wjrj].  The  estimators  <r2 (d)  and  a|(d) 
exhibit  the  following  properties:  (i)  They  are  both  biased  and 
(ii)  5|(d)  exhibits  smaller  MSE  from  the  true  value  than 
a\{d).  ■ 


Utilizing  Proposition  1,  estimators  for  the  filter-output  J- 
divergence  become  readily  available.  The  following  theorem 
identifies  their  relative  merits.  The  proof  is  included  in  the 
Appendix. 

Theorem  1:  Define  the  two  supervised  estimators  of  the 
output  J-divergence  Js,i(d)  =  and  Js,2(d)  = 

where  the  subscript  “S”  identifies  a  supervised  (pilot  assisted) 
implementation.  Then,  for  a  given  information  bit  pilot  se¬ 
quence  of  size  N  =  Ni  +  N2,  where  Ni  and  N2  are 
the  cardinalities  of  the  sets  {6(i)  = -1}  and  {b(i)  = +1} 
respectively,  both  estimators  are^biased  while  the  MSE  of 
Js,2(d)  is  less  than  the  MSE  of  Js,i(d),  i.e. 


E 


4y2(dy 
*>(d)  . 


where  ^  is  the  true  value  of  J(d). 


(14) 


Using  the  preferred  estimator  Js>2(c? ),  the  supervised  imple¬ 
mentation  of  the  output  J-divergence  AV  selection  rule  takes 
the  final  form  given  by  the  following  criterion. 

Criterion  2a:  For  a  given  information  bit  pilot  sequence 
of  size  Nf  the  supervised  J-divergence  AV-filter  selection  rule 
chooses  the  AV-filter  estimator  wj2  such  that 


d2  =  arg  max 


{dsAd)} 


=  arg  max  \ 
d 


4  |  £  E  K*)R*  [w| Ti] 
1  =  1 


N 


Jr  E  [Hi)Re  [wjrf]  -  p(d)}2 

i—1 


}  (15) 


2)  Unsupervised  (Blind)  Output  J-divergence  Rule 
The  blind  implementation  of  the  rule  is  obtained  by  sub¬ 
stituting  the  information  bit  6  in  (10)  by  the  detected  bit 
b  =  sgn  (Re  rj )  (output  of  the  sign  detector  that 


To  estimate  Js{d)  from  a  data  record  of  finite  size,  we 
substitute  the  expectations  in  (16)  by  sample  averages.  The 
following  criterion  summarizes  the  corresponding  AV-filter 
selection  rule. 

Criterion  2b:  For  a  given  data  record  of  size  N,  the  unsu¬ 
pervised  (blind)  J-divergence  AV-filter  selection  rule  chooses 
the  AV-filter  estimator  wj3  such  that 

d3  =  argmax(jB(d)}  (17) 


|  4  £  El-Re [wMI 

2 

> 

1  N 

r  n 

.  1 

"2 

UrE  l-Relw^]!2- 

V  t=l 

Jr  E  1-Re  [wjnll 

L  f=i 

J 

III.  Simulation  Studies 

We  revisit  the  signal  model  of  Fig.  2.  Using  the  same 
set-up  as  in  Fig.  2(a)  (i.e.  N  =  230,  SNRo  -  SdB)f 
we  examine  the  performance  of  the  AV-filter  selection  rules 
under  the  assumption  that  no  pilot  sequence  {£>(i)}»  of  the 
user  signal  of  interest,  user  0,  is  available.  In  Fig.  3,  we 
plot  the  empirical  probability  density  function  (pdf)  of  the 
differences  ^c?i  -  dopt  j  and  (d2  -  dopt^  where  d\  and  d3 
denote  selections  according  to  Criterion  1  and  Criterion  2b, 
respectively,  while  dopt  denotes  the  (“genie”  found)  maximum 
SINR  optimum  choice  of  the  number  of  auxiliary  vectors 
d  (as  in  Fig.  2,  we  conduct  1,000  experiments  with  100 
different  channel  realizations  and  10  independent  data  record 
generations  per  channel).  The  pdf’s  of  the  two  differences 
and  (dz-dop^j  shown  in  Fig.  3  capture  the 
level  of  failure  in  estimating  the  optimum  AV  parameter  d. 
The  heavy  concentration  of  both  pdf’s  around  0  implies  that 
both  criteria  provide  reliable  estimates  of  the  optimum  number 
of  auxiliary  vectors. 

In  Fig.  4,  we  plot  the  BER2  of  the  AV-filter  estimators  w j 
and  wj3  as  a  function  of  the  SNR  of  the  user  of  interest  for 
data  records  of  size  N  -  230.  The  BER  curve  of  the  “genie” 

2The  BER  of  each  filter  under  consideration  is  approximated  by 
Q  WSiNRnt)  [14]  since  the  computational  complexity  of  the  BER 
expression  for  this  antenna  array  CDMA  system  prohibits  exact  analytical 
evaluation. 
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assisted  BER  optimum  filter  choice  wdopl  as  well  as  the 
corresponding  curves  of  the  ideal  MMSE/MVDR,  SMI,  RAKE 
matched-filter  (MF),  and  multistage  nested  Wiener  filter  [9], 
[10]  with  the  preferred  number  of  eight  stages  [16]  are  also 
included  for  comparison  purposes.  We  observe  that  both  w .- 
and  wj3  are  very  close  to  the  “genie”  optimum  AV-filter  choice 
and  outperform  significantly  the  SMI,  RAKE  matched  filter,3 
and  multistage  nested  Wiener  filter  estimators.  We  also  observe 
that  for  moderate  to  high  SNR’s  of  the  user  of  interest,  the 
J-divergence  selection  rule  is  slightly  superior  to  the  CV-MOV 
selection  rule.  The  opposite  is  true  in  the  low  SNR  range.  This 
is  explained  by  the  fact  that  the  J-divergence  approximation 
J( w)  ~  «fjs(w)  used  in  Proposition  2  is  less  accurate  for  low 
filter  output  SINR  values.  On  the  other  hand,  for  high  filter 
output  SINR’s  the  discrimination  capability  of  the  CV-MOV 
rule  is  not  as  sharp. 

Finally,  Fig.  5  repeats  the  study  of  Fig.  4  as  a  function 
of  the  data  record  size  N.  The  SNR  of  the  user  signal  of 
interest  is  fixed  at  8 dB.  To  illustrate  the  benefits  that  we  gain 
by  actively  seeking  the  most  appropriate  number  of  AV’s  for 
each  given  input  data  record ,  we  include  the  BER  curve  of 
AV  filter  estimators  that  either  use  always  some  fixed  number 
of  auxiliary  vectors  (in  this  study  we  consider  d  =  15)  or  for 
each  data  record  size  N  use  dmeon(/V)  auxiliary  vectors  where 
dm  eon  (N)  denotes  the  integer  nearest  to  the  empirical  mean 
of  thejjdf  of  the  random  variable  dopt(N).  We  observe  that 
both  w(ii  and  outperform  w15  and  wdm„n;  of  course, 
dmean  cannot  be  calculated/estimated  in  practice  without  long 
pilot  signaling  (not  needed  by  either  one  of  the  dx  and  d3 
proposed  selection  rules). 

IV.  Summary  and  Discussion 

In  [2],  [3]  it  was  shown  that  for  a  given  finite  data  record 
the  sequence  of  AV-filter  estimators  initialized  at  the  matched- 
filter  converges  to  the  SMI  estimator  of  the  MMSE/MVDR 
filter.  It  was  also  noted  that  the  early  non-asymptotic  elements 
in  the  sequence  provide  favorable  bias/variance  balance  and 
outperform  in  MS  filter  estimation  error  the  SMI,  (constraint)- 
LMS,  RLS,  multistage  nested  Wiener  filter  (MNWF),  and  DL- 
SMI  filter  estimators.  In  this  paper  we  considered  the  problem 
of  identifying  the  best  AV-filter  estimator  in  the  sequence.  We 
proposed  two  selection  criteria,  one  based  on  the  filter  output 
variance  and  one  based  on  the  J-divergence  of  the  filter  output 
conditional  distributions.  The  first  AV-filter  selection  rule  aims 
at  the  minimization  of  the  estimated  filter  output  variance 
using  the  well  established  statistical  method  of  cross-validation 
(in  particular  “leave-one-out”  cross-validation).  This  selection 
criterion  can  be  applied  to  general  filter  estimation  problems. 
The  need  to  invoke  cross-validation  was  established  formally. 
The  second  AV-filter  selection  rule  is  tailored  to  applications 
that  can  be  formulated  as  binary  hypothesis  testing  problems 
defined  on  AV-filtered  data.  This  selection  rule  aims  at  the 

JWe  recall  that  the  ideal  MMSE/MVDR  filter  is  in  fact  the  AV  filter  Woo 
(d  oo  AV’s),  the  SMI  filter  is  (N),  and  the  RAKE  matched-filter  is 
w0  (d  =  0  AV’s).  As  a  side  note,  the  ideal  MMSE/MVDR  filter  (d  =  oo) 
lies  in  the  signal  subspace  (see  for  example  [17]),  while  both  the  SMI  filter 
(d  =  oo)  and  the  AV-filter  estimates  (0  <  d  <  oo)  do  not  necessarily  belong 
to  either  the  true  or  the  estimated  signal  subspace. 


maximization  of  the  estimated  J-divergence  between  the  filter- 
output  conditional  distributions  under  each  hypothesis.  We 
presented  a  supervised  and  a  blind  implementation  of  the  J- 
divergence  selection  criterion  for  problems  where  the  received 
data  correspond  to  binary  antipodal  signals  in  noise. 

Simulation  studies  examined  and  compared  the  operational 
characteristics  of  the  proposed  methods.  With  respect  to  the 
relative  merits  of  the  output  J-divergence  and  the  cross- 
validation  minimum  output  variance  selection  rule,  we  ob¬ 
served  that  for  moderate  to  high  output  SINR’s  the  former 
method  appears  somewhat  superior  to  the  latter  (for  high 
SINR’s  the  cross-validated  minimum  output  variance  rule  is 
not  as  sharp  in  discrimination  ability).  In  contrast,  in  low 
output  SINR  the  J-divergence  method  is  slightly  lagging  in 
performance  (technically,  the  approximation  in  Proposition  2  is 
less  accurate  for  near  0 dB  or  lower  filter  output  SINR  values). 


Appendix  A 

Proof  of  Proposition  1 

Proof:  The  quantities  ££&  (p~ -p~(d))2  and 

Sjii  (p'j  -  fi+(d))2  are  the  ML  estimators  of  the  filter 
output  variance  conditioned  on  b  =  -1  and  b  =  -fl, 
respectively  [15,  p.  179].  Both  estimators  are  biased.  (In  fact, 
their  unbiased  counterparts  that  have  multiplying  factors  — 1 
and  instead  of  7^  and  ■£,  respectively,  exhibit  higher 
MSE).  The  MSE’s  of  the  estimators  of  interest  d\(d)  and 
a2  (d)  are  as  follows: 


and 


£{( 

d\{d)  -cr2(d))2} 

4 

N\o2{d)  V 

u 

\  n1  +  n2 

Ni  +  N2) 

+E< 

/EfiW-iiW 

N2o2{d) 

[{  Ni+N2 

Ny+N2 

+2x 

Ffn?=i(p7  ~P~(d))' 

2  Nio2(d) 

\  nx  +  n2 

Ni+N2 

xe| 

e -£+(d))2 

N2a2{d)  \ 

Ni+N2 

N!+N2  ] 

(18) 


=  (2 Ni  ~  1)  <rA(d)  +  (2AT2  -  1)  o4(d)  +  2o4(d) 
(^i+JV2)2 

2cr4(d) 

~  N2  +  N2 

MSEn{d)  =  E  | (d%(d)  -  <r2(d))2} 

ftM  +  JVa)-  1] 

(N!+N2)2 


(19) 


Thus, 


MSEsllJ)-MSEs,m  =  ^y>0. 


(20) 
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Appendix  B 
Proof  of  Theorem  1 

Proof:  and  o2~  (d)  are  independent  random 

variables  with  distributions  Af  fi(d ),  j  and 

~Jf(dj£2~(d)  ~  X%i-i  [18]  (Xatx_i  denotes  the  chi-square 
distribution  with  JVi  -  1  degrees  of  freedom).  Similarly, 
fi+(d)  and  o2+(d)  are  independent  random  variables  with 
distributions  fl+{d)  ~  Af  and  ^j52+(d)  ~ 

Xn2- i>  Furthermore,  (p~(d),o2~(d))  and  (p.+(d),o2+(d)) 
are  mutually  independent  because  (jl~(d),o2~(d))  and 
(fi+(d),d2+(d))  are  evaluated  from  two  independent  training 
sets.  Since  fi(d)  =  -  ^p+{d)  and  5?(d)  = 

jfd2~(d)  +  jfo2+(d),  it  is  implied  that  /2(d)  and  o2(d) 
are  independent  random  variables  with  distributions  p(d)  ~ 
■V  (p(d),  £-^)  and  91(d)  ~  \n-2  where  N  =  Ni  + 
N2  is  the  size  of  the  given  pilot  bit  sequence.  Therefore, 


$2{d)  ~  Xn-v  respectively.  Therefore, 

=  _JL_  +  _L_ 

o2{d)  N-3  N-3' 

prj2  lQ[N2n*(d)+Wfj,2(d)o2(d)+3o4(d)}  .... 

{  S,2\  )>  (T4(d)  (JV  -  3)  (N  -  5)  ■  (27) 


Thus,  the  MSE  of  Js,2(d)  is 


£{Js,i(d)}  =  4E& 


!(d)i£{??W} 


_  4  \°JM  .  ..2/ j\1  N 

4  [  N  (^J  o2(d)  (AT  —  4) 

_  V(d)  N  4 
o2(d)  JV-4  +  JV-4’ 

_  16\M2fJ.4(d)+GNp2(d)o2(d)+3o4(d)] 
o*{d)  (JV  -  4)  (JV  -  6) 


*{w)}  ~ 

E{^(d)}  “  o' 


Thus,  the  MSE  of  Js,i(d)  is 


JV 

2(d )  (AT  — 4)’ 
N2 


'(d)  (JV-  4)  (JV-6)' 


16a _ 166_  16//4(d) 

(JV  —  4)  (AT  —  6)  JV-4  +  a4(d)  U' 


where 


a  t  U (d)  +  ^2(d)g2(d)  +  gfMl  and 

M  '  '  W  T  J\?i  J  ^(d) 


■r^M rw- 


=E{ 


JsMd)  - 


4n2(d) 


=  16a  16b  16/(d) 

(iV  -  3)  (JV  —  5)  JV-3  +  cr4(d) 

where  the  first  and  second  inverse  moments  E  {  1  and 

E  }  ex^st  ^or  N  >  3  and  iV  >  5,  respectively,  and  a 
and  b  are  defined  above.  From  (25)  and  (28),  assuming  AT  >  6 
which  is  the  condition  for  all  inverse  moments  to  exist,  we 
obtain 

MSEi  -  MSE2 

_  16q _ 16a _ L  b  b 

( JV-4)(JV-6 )  (JV— 3)(JV— 5)  i‘  N-3  N- 4 

—  16  r  2  AT— 9  _  i 

~  (JV— 3)(JV— 4)  |_(JV— 5){JV— 6)  “  0 

^  16  T  2(N-5)  _  i 

^  (JV— 3)(JV— 4)  |_(JV-5)(JV-6)a  “  0 

_  32l6Nft*(d)+3N/P(d)^(d)+6tA(d)v3 (d) +3<r4 (d)l  ^  „  .... 


The  inverse  moments  and  E  exist  for 

N  >  4.  and  JV  >  6,  respectively,  and  are  given  by  [19] 


(JV— 3)  (JV— 4)(JV— 6)<r4  (d) 


>  0.  (29) 


Appendix  C 

Proof  of  Proposition  2 
Proof:  Define  Y  =  |ile(w^r)|.  The  pdf  of  Y  is 

V2na(d)  V2^a(d) 

where  U(y)  is  the  unit  step  function.  The  mean  of  Y  is 

E(Y)  =  j  yfY{y)dy 


The  series  expansion  of  Q(x)  is  [20] 


y/2nx  \ 


(-!)”!  •  3  -  •  •  (2n— 1)] 


On  the  other  hand,  y{d)  and  o2{d)  are  independent  random 
variables  with  distributions  /2(d)  rsj  and 


where  Rn 


(~l)n+1l 


3-  ••  (2n  -|- 


■  exp  (-y)cJt  is  the  remainder  which  is 
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alwavs  less  (in  absolute  value)  than  the  first  neglected  term. 
If  »  1,  (31)  can  be  written  as 


« (33) 


The  variance  of  Y  is 


Var(Y)  =  E(Y2)  -  E2(Y)  =  fi2(d)  +  a2(d)  -  E2(Y)  (34) 
and  using  (33)  we  may  approximate 

Var(Y)  *  <r2(d).  (35) 

Finally,  from  (33)  and  (35)  we  obtain 


JB(d)  = 


4  E2(Y) 
Var(Y) 


a2(d) 


=  J(d). 


(36) 
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Input: 

Autocovariance  matrix  R,  constraint  vector  v, 
desired  response  wH  v  - 1 

Initialization: 


w  *—  y 

w°  ilvii3  • 

Iterative  computation: 

For  d=1,  2,  ...  do 
begin 


W" 

9d  :=  R  Wd-i  ~  j^jj2  R  Wd-i 
If  gd  =  0  then  EXIT 


fid 


9d  R  w 


dzi 


end 


9dHR  gd 

wd  :=  w<M-ndgd 


Output: 

Filter  sequence  w0l  w1f  w2 . 


Fig.  1 .  The  “AV  algorithm”  for  the  iterative  generation  of  the  filter  sequence 
wo,wi,w2,...  . 


(a)  N  =230,  SNRo=8dB 


Fig.  2.  Histogram  of  the  number  of  AV’s  d  utilized  by  the  best  (max  SINR) 
AV-filter:  (a)  N  =  230,  SNRq  =  8 dB,  (b)  N  =  363,  SNRq  =  8 dB,  and 
(c)  N  =  230,  SNRq  =  4 dB. 
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Abstract — We  investigate  the  coarse  synchronization  per¬ 
formance  of  blind  adaptive  linear  self-synchronized  receivers 
for  asynchronous  direct-sequence  code-division  multiple-access 
communications  under  finite  data  record  adaptation.  Based 
on  transformation  noise  modeling  techniques,  three  alternative 
methods  are  developed,  leading  to  analytical  expressions  that 
approximate  the  probability  of  coarse  synchronization  error  of 
matched-filter-type  and  minimum-variance  distortionless-re¬ 
sponse-type  receivers.  The  expressions  are  explicit  functions  of 
the  data  record  size  and  the  filter  order  and  reveal  the  effect  of 
short  data-record  sample  matrix-inversion  implementations  on 
the  coarse  synchronization  performance.  Besides  their  theoretical 
value,  the  derived  expressions  provide  simple,  highly-accurate 
alternatives  to  computationally  demanding  performance  evalua¬ 
tion  through  simulations.  The  effect  of  the  data  record  size  on  the 
probability  of  coarse  synchronization  error  is  further  quantified 
through  the  use  of  a  receiver  synchronization  resolution  metric. 
Numerical  and  simulation  studies  examine  the  accuracy  of  the 
theoretical  developments  and  show  that  the  derived  expressions 
approximate  closely  the  actual  coarse  synchronization  perfor¬ 
mance. 

Index  Terms — Adaptive  filters,  approximation  methods,  code¬ 
division  multiple  access  (CDMA),  finite  data  analysis,  minimum 
mean-square  error  (MMSE)  filtering,  spread-spectrum  communi¬ 
cation,  synchronization. 


I.  Introduction 

THE  effectiveness  of  a  receiver  designed  for  rapidly 
changing  wireless  direct-sequence  code-division  mul¬ 
tiple-access  (DS-CDMA)  communication,  environments 
depends  on  the  following  design  attributes:  1)  low  compu¬ 
tational  complexity;  2)  multiple-access  interference  (MAI) 
near-far  resistance;  and  3)  system  adaptivity  with  superior 
performance  under  limited  data  support.  Adaptive  short 
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data-record  designs  appear  as  the  natural  next  step  [  1  ]— [4] 
to  a  matured  discipline  that  has  extensively  addressed  the 
first  two  design  objectives  in  ideal  setups  (perfectly  known 
or  asymptotically  estimated  statistical  properties)  [5],  [6]. 
System  adaptivity  based  on  short  data  records  is  necessary  for 
the  development  of  practical  receivers  that  exhibit  superior 
bit-error  rate  (BER)  performance  when  they  operate  in  rapidly 
changing  communication  environments  that  limit  substantially 
the  available  data  support. 

In  Part  I  [7],  we  considered  self-synchronized  receivers  (in¬ 
tegrated  synchronizers/demodulators)  and  we  presented  three 
schemes  along  the  classical  receiver  evolution  path.  We  started 
with  a  matched-filter-type  (MF)  structure,  we  continued  with 
a  minimum- variance  distortionless-response- type  (MVDR) 
scheme  and,  finally,  we  developed  an  auxiliary-vector-type 
(AV)  alternative,  all  of  order  either  twice  or  equal  to  the 
system  spreading  gain  L.  Simulation  studies  illustrated  the 
performance  of  the  proposed  structures  in  terms  of  BER  versus 
data  support  for  a  given  signal-to-noise  ratio  (SNR)  of  the  user 
of  interest  or  in  terms  of  BER  versus  SNR  for  a  given  data 
record  size. 

In  this  paper,  we  develop  three  methods  for  the  analysis  of  the 
finite  data-record  behavior  of  self-synchronized  linear  receivers 
and,  in  particular,  the  effect  of  the  filter  order  and  the  data-record 
size  on  the  coarse-synchronization  error  rate  (CSER).  We  derive 
analytical  expressions  that  approximate  closely  the  probability 
of  coarse  synchronization  error  of  MF-type  and  MVDR-type 
schemes.  Probability  of  coarse-synchronization  error  analysis 
for  AV-type  schemes  based  on  a  finite  data-record  size  is  pro¬ 
hibitively  complex,  and  thus,  not  attempted  at  this  time. 

The  paper  is  organized  as  follows.  In  Section  II,  we  present 
the  system  model  and  we  summarize  briefly  the  combined  syn¬ 
chronization/demodulation  algorithms  developed  in  [7].  In  Sec¬ 
tion  III,  we  develop  analytical  expressions  that  approximate  the 
coarse-synchronization  error  probability  (Pc se),  while  in  Sec¬ 
tion  IV,  we  derive  a  sequence  of  increasingly  tight  lower  bounds 
on  Pcse.  In  Section  V,  we  discuss  the  effect  of  the  filter  order  and 
the  data-record  size  on  the  CSER.  The  accuracy  of  the  analyt¬ 
ical  approximations  of  Sections  III  and  IV  is  examined  through 
simulations  in  Section  VI.  Some  final  conclusions  are  drawn  in 
Section  VII. 

II.  System  Model  and  Algorithmic  Description 

We  consider  an  asynchronous  DS-CDMA  system  populated 
by  K  active  users  transmitting  over  a  common  additive  white 
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Fig.  1 .  Coarse  synchronization  based  on  filters  of  order  I . 


Fig.  2. 


Construction  of  the  sequences  /  =  0 _ ,L  —  1. 


Gaussian  noise  (AWGN)  channel.  The  received  signal  r(t)  is 
the  superposition  of  the  K  transmissions  corrupted  by  channel 
noise,  i.e., 

K—  1  oo 

rW  =  E  E  yMbk^Skit  -iT-  Tic)  +  n(f).  (1) 

k=Q  i=-oc 

In  the  above  expression,  with  respect  to  the  kih  user,  rk 
is  the  signal  delay  relative  to  the  receiver’s  time  reference 
point,  bk(i)  €  {-1,  +1}  is  the  ith  information  bit,  Ek  is  the 
transmitted  energy,  T  is  the  information  bit  period,  and  n(t)  is 
AWGN.  The  normalized  signature  waveform  Sk(t)  assigned  to 
the  A:th  user  has  the  form 

L-l 

Sk(t)  =  '52sk(l)PTc{t-lTc),  k  =  0,...,K-l  (2) 

1=0 

where  Tc  is  the  chip  duration,  Ptc{ •)  is  a  rectangular  pulse 
with  support  [0,TC],  L  =  T/Tc  is  the  system  spreading  gain, 
and  sk(l)  is  the  /th  element  of  the  normalized  bipolar  signa¬ 
ture  or  spreading  vector  s*  =  [sfc(0),  sk(  1), . . . ,  sk(L  -  1)]T 
that  uniquely  identifies  the  fcth  user.  Without  loss  of  generality, 
we  assume  that  rk  £  [0,T),  k  =  0, ...,K-  1.  Thus,  we 
may  write  the  delay  rk  as  a  sum  of  an  integer  multiple  of  chips 
plus  a  fraction  of  a  chip:  rk  =  (nk  +  6k)Tc  where  nk  £ 
{0, . . .  ,Z  —  1}  and  Sk  £  [0, 1).  After  chip-matched  filtering 
and  chip-rate  sampling,  the  continuous-time  signal  r(t)  in  (1) 
is  converted  to  a  discrete-time  sequence  {r(n)}^°=_0O,  where 

<+»>  = 

In  this  paper,  we  analyze  the  coarse  synchronization  perfor¬ 
mance  of  L  and  21,-order  algorithms  of  MF  and  sample  ma¬ 
trix-inversion  (SMI)  MVDR  type.  A  brief  description  of  the 


coarse  synchronization  step  of  these  algorithms  follows  (details 
can  be  found  in  [7]).  Fig.  1  shows  the  block  diagram  representa¬ 
tion  of  an  Z-order  coarse  synchronizer/demodulator.  The  buffer 
follows  a  chip-rate  sampler  and  groups  the  received  samples  into 
L  sequences  of  vectors,  {rfjj  £  /  =  0, . . . ,  L  -  1,  as  de- 

picted  in  Fig.  2  where 

ri°  =  ir(iL  +  0, r(iL  +  1+1),..., r((i  +l)£  +  l-  i)ft 

1  =  0,...,  L-l.  (3) 

In  the  case  of  2L-order  algorithms,  the  received  samples 
{r(n)}„  are  grouped  into  a  sequence  of  overlapping  vectors, 
r<  £  SR2t,  obtained  by  a  sliding  window  such  that  successive 
vectors  share  L  common  data  samples 

fi  =  [r(iL),  r(iL  +  1), . . . ,  r((i  +  2 )L  -  1)]T .  (4) 

To  facilitate  our  presentation,  overlined  variables  always  refer  to 
2Z-order  processing  to  distinguish  themselves  from  the  corre¬ 
sponding  variables  used  by  Z-order  algorithms.  Table  I  provides 
the  definition  of  both  the  ideal  and  estimated  decision  statistic 
(filter  output  energy)  of  each  filter  under  consideration,  namely 
SMI-MVDR  and  MF  of  order  Z  or  2 Z.  Table  II  provides  addi¬ 
tional  definitions  needed  in  the  expressions  of  Table  I. 

For  the  Z-order  case  of  Table  I,  R(')  4  E  j  is 

the  input  covariance  matrix  and  R/'>  =  1/N  J^iLo1  r-°r^)T 
is  its  sample  average  estimate  (N  is  the  total  number  of 
samples  or  data-record  size).  Similarly,  for  the  2Z-order 
case,  R  f  EfarJ}  and  fi.  =  l/N^1  f,rj.  In 

Table  II,  s0  =  (0, so(0),...,so(Z  —  2)]T  denotes  the 
one-chip  right-shifted  zero-filled  version  of  s0,  while  s^, 
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TABLE  I 
Decision  Statistic 


L-order 

SMI-MVDR 

Ui  ~  (dPII  R(L,/2J)  ’4ll) 

% = (dSiR,li,2|r’dSiF1 

MF 

cf,=djji]rR^j)dji> 

2L-order 

SMI-MVDR 

Ui  =  (dU0||  R  *  d[ii>||) 

tj .  _  fjb)7  p  1  j(i) )  1 

uj  -  (djjou  R  d!!0|jj 

MF 

Uj  =d|f0||  R  dfjili 

ff , j (i)T  p  j0) 

Uj  -d||0||  R  djjou 

TABLE  II 

Effective  Signature 


L- order 

d!|o||  “ 

|  S||o||(0),  if  j  is  even 
\  S||0||(0.5)  if  j  is  odd, 

2L-order 

j(i)  a  1 

dnoir  | 

ijr  (°)>  3  is  even 

.  §l|o||  2J)  (°-5)  if  J  ^  odd. 

11011  ll(I-^)50)+<^+l)|| 

l  =  0, . . . ,  L  —  1,  is  the  /-shifted  version  of  the  zero-padded 
2L-long  vector  [sj,  0, . . . ,  0]r. 

An  SMI-MVDR-type  or  MF-type  coarse  estimate  of  the  delay 
to  can  be  determined  as  follows: 

fri  =  /  ( )  To  ^ imax  is  even 

l  (  +  °-5)  TC  if  Jmax  is  Odd 

where 

Jmax  =  arg max  {Uj\j  =  0, . . . ,  2L  -  1  j  (6) 

and  Uj  is  obtained  from  Table  I  for  each  filter,  respectively. 


III.  Finite  Data-Record  Performance  Analysis 


The  probability  of  coarse  synchronization  error  is  defined  as 
the  probability  that  the  coarse  synchronization  algorithm  fails  to 
provide  an  estimate  within  the  “pull-in”  range.  In  this  paper,  we 
adopt  the  commonly  used  assumption  that  the  pull-in  range  is 
equal  to  (1/2)TC  around  the  correct  delay  To.  However,  the  per¬ 
formance  analysis  presented  in  this  section  can  be  carried  out  with 
straightforward  modifications  for  any  given  pull-in  range  value. 
The  probability  of  coarse  synchronization  error  is  given  by 


1  —  Pr 


\T0  -  To|  <  -Tc 


(7) 


probability  of  coarse  synchronization  error  will  be  equal  to 

Pc se  =  1  -  Pr  [jmax  €  W(t0)]  .  (8) 

Different  choices  of  the  pull-in  range  can  be  accommodated  by 
appropriately  modifying  the  definition  of  H(ro).  In  the  rest  of 
this  section,  we  derive  close  approximations  of  the  coarse  syn¬ 
chronization  performance  of  the  four  algorithms  under  consid¬ 
eration. 


A .  MF-Type  Receivers  (2 L  Order) 


We  recall  that  in  this  case,  Uj  =  Rdy^i  and  Uj  = 

aJST  AajjSi,.  3  =  o,  l, . . . ,  2L  -  1.  Then,  the  CSER  p£e% 
of  the  2i>order  MF  algorithm  can  be  expressed  as  the  prob¬ 
ability  that  the  index  of  the  largest  decision  variable  Uj,  j  = 
0, 1, . . . ,  2£  —  1,  is  not  contained  in  H(to );  that  is 

PcseMF  =l  -  Pr  arg  max  {Uj\j  =  0, . . . ,  2L  -  1  j  €  H(t0) 

(9) 


,0) 


=1  -  Pr[max  €  H(tq)  j 
=  0, 


>  rnax^i 


.,2L-l,jtn(T0) 


}]• 


The  latter  form  of  P^eMF  IS  particularly  easy  to  handle, 
as  we  will  see.  We  observe  that  the  evaluation  of  the  joint 
probability  density  function  (pdf)  of  the  random  quantities  Uj, 
j  =  0, . . . ,  2L  - 1,  requires  knowledge  of  the  pdf  of  the  random 
matrix  R.  We  recall  that  R  is  the  sample  average  estimate  of  the 
autocorrelation  matrix  R  of  the  received  vectors  r,  which  are 
distributed  according  to  a  mixture  of  up  to  23K  Gaussian  distri¬ 
butions,  and  they  are,  by  construction,  statistically  dependent. 
Unfortunately,  the  use  of  exact  closed-form  expressions  for  the 
pdf  of  the  sample  covariance  matrix  that  is  formed  by  vector 
samples  drawn  from  a  Gaussian  mixture  distribution  [8]  leads 
to  mathematically  intractable  expressions  for  the  probability 
of  synchronization  error.  An  additional  complicating  factor  for 
the  analysis  of  the  finite  data-support  behavior  of  the  MF-type 
(and  SMI-MVDR-type)  receivers  is  the  underlying  dependence 
of  the  vector  samples.  In  that  respect,  we  make  the  following 
simplifying  assumptions.  We  assume  that  the  received  vectors 
are  uncorrelated  and  identically  Gaussian  Af( 0,R)  distributed 
[thus  independent  and  identically  distributed  (i.i.d.)]  [9], 
[10].  The  latter  Gaussianity  postulation  can  be  considered  as 
a  wishful  approximation  of  the  true  distribution,  while  the 
former  independence  postulation  can  be  justified  if  we  consider 
“guard”  bands  of  size  L  between  the  received  vectors,  so  that 
successive  input  vectors  do  not  share  common  information  bits. 
The  effect  of  the  above  assumptions  will  be  examined  in  the 
simulations  section  through  comparisons  with  the  exact  Pcse • 

It  is  known  [11]  that  if  Xo,Xi, . . .  ,xjv-i  €  are 
i.i.d.  A/'fO,  S)  vectors,  then  the  distribution  of  the  matrix 

x  =  is 


where  the  coarse  estimate  of  fo  is  given  by  (5)  and  (6).  We  recall 
that  jmax  in  (5)  and  (6)  is  the  index  of  the  filter  with  the  highest 
output  energy.  If  we  define  7f(r0)  as  the  set  of  the  filter  indexes 
that  yield  a  timing  estimate  f0  within  (1/2)TC  about  to,  then  the 


/(X)  = 


|X|(Ar-p-i)/2exp{-itrj;-1x} 

2pN/2%p(p-l)/4\£\N/2  J}P=i  r  I- 


p  >  N 

(11) 
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where  F(-)  is  the  Gamma  function.  The  distribution  in  (11) 
is  called  the  Wishart  distribution  with  N  degrees  of  freedom 
(DOFs)  and  is  denoted  by  WP(E\  N).  A  random  matrix  dis¬ 
tributed  according  to  ( 1 1)  is  called  a  Wishart  matrix.  In  our  case, 
R  is  approximately  Wishart  with  distribution  W2l(R/N;  N ). 
If  we  define  the  matrix 

D4/d(0)  d(1)  d(2L-!)>i 

u  vaiioipdiioir-^diioii  ;  (12) 

then  the  random  variables  Ujt  j  =  0. . . . ,  2L  -  1,  form  the 
main  diagonal  of  DTRD.  For  a  full-rank  matrix,  D,  DrRD 
is  Wishart  W2^((l/iVr)DrRD;7V’)  [12],  and  the  main  diag¬ 
onal  elements  Uj,  j  =  0, . . . ,  2L  -  1,  are  jointly  distributed 
according  to  a  multivariate  Gamma  distribution  [13].  However, 
D  is  not  full  rank  in  general,  and  even  if  it  were,  it  would  have 
been  of  little  help  since  closed-form  expressions  of  the  multi¬ 
variate  Gamma  pdf  do  not  exist  except  for  some  special  cases 
[14].  Thus,  appropriate  approximations  of  the  joint  pdf  of  the 
diagonal  elements  of  DTRD  with  tractable  multivariate  inte¬ 
grals  are  needed. 

Motivated  by  the  transformation  noise-modeling  techniques 
used  in  [15],  we  approximate  the  joint  pdf  of  the  MF  decision 
variables  Uj,  j  =  0, . . . ,  2L  -  1,  by  the  joint  pdf  of  a  non¬ 
linear  transformation  of  Gaussian  random  variables  Yj,  j  = 
0, . . . ,  2Z>  1.  The  nonlinear  transformation  and  the  covariance 

of  Yj,  j  =  0, . . . ,  2L  —  1,  are  chosen  such  that  the  outputs  of  the 
nonlinear  transformations  have  identical  covariances  with  the 
corresponding  Uj,  j  =  0, . . . ,  2L  -  1,  and  marginal  pdfs  that 
are  close  approximations  to  the  pdfs  of  Uj ,  j  =  0, . . .  ,  2 L  -  1. 
The  following  lemma  identifies  this  nonlinear  transformation 
and  the  statistics  of  the  random  variables  Yj,j  =  0, . . . ,  2L  - 1 . 
The  proof  is  included  in  the  Appendix. 

Lemma  I:  The  pdf  of  Uj  can  be  approximated  by  the  pdf  of 
UjY? ,  j  =  0, . . . ,  2L  -  1,  where  the  random  variables  Yj ,  j  = 
0, . . . ,  2Zr  -  1,  are  uncorrelated  Gaussian  with  mean  1  -  2/9 N 
and  variance  2/9 N.  ■ 

The  probability  of  coarse  synchronization  error  can  now 
be  evaluated  as  follows.  Let  the  random  variables  Zj, 
j  =  0, . . . ,  2L  —  1  be  defined  as 

Zj  =  g WZ2U}/3yi’  j  =  0,...,2L  -  1.  (13) 

It  is  straightforward  to  show  that 
Pr  ^max  j?7fc|A:  €  W(r0)| 

>  max  =0,...,2L-  l,j  t  H(r0)}] 

*Pt  [max{l4yfc3MW(T0)} 

>  max  {UjYf\j  =  0, . . . ,  2L  -  1,  j  #  7f(r0)}] 


The  random  variables  Zj,  j  =  0, . . . ,  2L  -  1,  are  uncorre¬ 
lated  Gaussian  (thus,  independent)  with  mean  Uj  =  l/j^3  and 
variance  Uj  =  (18N/(9N  -  2)2)u]/3.  The  pdf  of  Zj,  j  = 
0, . . . ,  2L  -  1  is 


where  g{z)  =  1/v^tt  exp(-z2/2)  is  the  pdf  of  a  Gaussian 
random  variable  with  mean  0  and  variance  1,  while  the  cumula¬ 
tive  distribution  function  (cdf)  of  Zj  is 

^z)ss0{L^L)  (16) 

where  Q(z)  is  the  cdf  of  a  Gaussian  random  variable  with  mean 
0  and  variance  1.  The  cdf  of  max{Zj \j  =  0, . . . ,  2L  -  1,  j  £ 
W(t0))  is  given  by  [17] 

2L-1 

^max,\?<(ro)(2)  —  JJ  j(z)  (17) 

where  the  index  “\?f(ro)”  indicates  that  the  maximum  is  taken 
over  a  set  that  does  not  include  the  elements  of  7<(to).  There¬ 
fore,  the  corresponding  pdf  is  given  by 

2Z/  —  1  /  2L—1  \ 

/max,\tt(r0)(z)  ~  I  II  ^j(z)  J  * 

l=OJen(T0)  \  J 

(18) 

Similarly,  the  cdf  and  pdf  of  max{Zjt|fc  G  W(r0)}  are  given 
by 

^max/Hfro)^)  =  JJ  (z)  (19) 

j€H(r0) 

and 

/max,'H(ro)  (^)  =  [  fl(z)  JJ  (20) 

*€W(ro)  \  jen(To)jjtl  J 

respectively.  Since  Zj,  j  =  0, .  ..,2X  -  1,  are  independent, 
the  two  random  variables  max{Zj|j  =  0, ... ,2L  -  l,j  £ 
W(ro)}  and  max{Zk\k  G  H{% o)}  are  also  independent,  there¬ 
fore,  the  pdf  of  their  difference  max{Zj|jf  =  0, . . . ,  2L  - 1,  j  £ 
«(r0)}  -  max{Zfc|fc  G  W(r0)}  is  given  by 

/+OC 

/ max,\7t(ro)(^  “  2/)/max,7i(r0)(“^2/)^2/-  (21) 

•OO 

We  conclude  that  the  probability  of  coarse  synchronization  error 
for  the  21,-long  MF-type  coarse  synchronization  algorithm  is 


=  Pr  [max  {Zk\k  €  W(r0)} 

>  max  {Zjjj  =  0,...,2L-  l,j  0  H(r0)}] 

=  Pr  [max  {Zj\j  =  0, . . . ,  21  -  1,  j  £  H{t0)} 

—  ineix  {Zk\k  6  W(tq)}  <  0] .  (14) 


-1  -  /  fdiff(z)dz 

J  —  DO 

/0  A-f  oo 

/  / max,\X(r0)(^  “  V ) 

•OO  j  “OO 


^  /max,  n(r0)(-y)dydz 


(22) 

(23) 
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/0  r+oo  2L— 1  / 

/  E  ( /»(*  - ») 

•oo  J- oo  [f=0,(g7<(ro)  \ 

2£-l  \' 

x  II  -  v) ) 
x  II  •?7j(_^)]  dvdz- 

t€«(r0)  \  jGH(r0),i#J  / 

(24) 

Expression  (24)  approximates  the  probability  of  coarse  synchro¬ 
nization  error  as  a  double  integral. 

B.  MF-Type  Receivers  (L  Order) 

In  the  case  of  the  MF-type  coarse  synchronization  al¬ 
gorithm  that  utilizes  filters  of  order  L,  the  probability  of 
coarse  synchronization  error  Pc^Mf  can  evaluated  by 
(24),  where  fj(-)  and  J7^-)  denote  the  pdf  and  cdf,  re¬ 
spectively,  of  Zj,  j  =  0, 1,  ...,2L  -  1  ( Zj  is  defined 
as  in  (13),  with  the  difference  that  Uj  is  now  given  by 

Uj  =  =  0,1,..., 21-  1).  The  proof 

follows  similar  reasoning  as  in  Section  III- A,  with  the  differ¬ 
ence  that  Uj  —  d||JQ||  R(&/2J)d|jjj||,  j  —  0, 1, . . . ,  2L  —  1  and 
R(Lj/2J)  is  wishart  Wl((1/N)RM/V);N)  with  N  DOFs. 

C.  SMI-MVDR-Type  Receivers  (2 L  Order) 

The  MVDR-type  SMI-implemented  coarse  synchro¬ 
nization  algorithm  that  utilizes  filters  of  order  2  L 
can  be  analyzed  in  a  similar  manner.  In  this  case, 

Uj  is  given  by  Uj  =  \  j  = 

0,1,..., 2L  —  1.  If  we  define  the  random  variables  Zj  = 
(9(JV  -  2L)/(9(N  -  2  L)  -  2  ))Uj/3YjJ  =  0, 1, . . .  ,2L  -  1, 
where  the  random  variables  Yj,  j  =  0, 1, . . . ,  2L  -  1,  are  un¬ 
correlated  Gaussian  with  mean  1  —  2/9(iV  —  2 L)  and  variance 
2/9 (N  —  2 L),  then  Zj ,  j  =  0, 1, . . . ,  2L  —  1  are  uncorrelated 
Gaussian  random  variables  with  mean  pj  =  U^3  and  variance 
<rf  =  Uj/3{18(N  -  2L)/[9(N  -  2 L)  -  2]2).  The  probability 
of  coarse  synchronization  error  ^cse,MVDR  *s  Slven  by  (24), 
where  Pj(')  and  fj(-)  denote  the  cdf  and  pdf  of  the  random 
variable  Zj ,  j  —  0, 1, . . . ,  2L  —  1.  The  proof  again  follows 
similar  reasoning  as  in  Section  III-A,  with  the  difference  that 

Uj  =  ft  dg|)  ,  j  =  0, 1, . . . ,  2L  —  1,  and  ft  is 

Wishart  W2i((l/N)ft;  iV)  with  N  DOFs.  The  latter  implies 
that  Uj  can  be  expressed  as  a  multiple  of  a  x2  random  variable 
with  N  -  2L  DOFs. 

D.  SMI-MVDR-Type  Receivers  (L  Order) 

For  the  MVDR-type  SMI-implemented  coarse  synchro¬ 
nization  algorithm  that  utilizes  filters  of  order  L,  we  have 

Uj  =  (d|ffR(L’72J)"ldfS)'1.  3  =  0,1,..., 2£  —  1. 
Then,  the  probabilty  of  coarse  synchronization  error 
A^Imvdr  is  sti11  given  by  (24),  where  Tj(-)  and 
fj(-)  denote  the  cdf  and  pdf  of  the  random  variable 


Zj  =  (9(AT  -  L)/(9(N  -  L)  -  2))U]/zYj,j  =  0, . . .  ,2L-1. 
The  random  variables  Yj,  j  =  0,1,...,  2L  -  1,  are  un¬ 
correlated  Gaussian  with  mean  1  -  (2/9 {N  -  L))  and 
variance  (2/9(iV  -  L))  therefore,  the  random  variables 
Zj,  j  =  0,1,...,  2L  -  1  are  uncorrelated  Gaussian 

random  variables  with  mean  pj  =  U^3  and  variance 

a)  =  U]/Z{18(N  -  L)/[9(N  -  L)  -  2]2).  The  proof  follows 
similar  reasoning  as  in  Section  III-A,  with  the  only  difference 

that  Uj  is  now  given  by  Uj  =  ^d[j^|| 

and  R(L7/2J)“l  is  wishart  ((i/iV)R,( Li/2!)"1 ;  2\f)  with  N 
»DOFs.  The  latter  implies  that  Uj  can  be  expressed  as  a  multiple 
of  a  x2  random  variable  with  N  ~~  L  DOFs. 

IV.  A  Recursive  Method  for  the  Approximation  of  the 
Probability  of  Coarse  Synchronization  Error 

In  this  section,  we  develop  a  recursive  procedure  for  the  eval¬ 
uation  of  the  probability  of  coarse  synchronization  error,  Pcse. 
The  method  evaluates  Pcse  as  a  sequence  of  increasingly  tight 
lower  bounds  on  the  approximate  expression  (24)  and  is  compu¬ 
tationally  simpler  than  the  latter  expression  that  evaluates  Pcse 
as  a  convolution  of  pdfs.  We  recall  that  (24)  with  appropriate 
definitions  of  fj(-)  and  Pj{-)  can  be  used  for  the  evaluation  of 

PcseNF'  Pcse?MF'  ^is^MVDR’  an(i  ^cs^MVDR  aS  discussed  in 
Section  III. 

We  begin  with  the  observation  that  the  cardinality  of  the  set 
W(to)  is  almost  always1  equal  to  two.  Thus,  without  loss  of  gen¬ 
erality,  we  assume  that  H(tq)  =  {0, 1}.  In  addition,  we  assume 
that  the  random  variables  Zj,  j  =  2, . . . ,  2L  -  1  are  ordered  in 
terms  of  decreasing  variance,2  i.e.,  Var {Zj}  >  Var{Zj+i}  for 
j  =  2, .. .  ,2L  —  2.  Let 

Pcse  =1  “  Pr  ^argmax{Zj|i  =  0, 1, . . . ,  k  + 1}  j  6  W(r0) 

(25) 

=1  -  Pr[max{Z0,  Zx}  >  ma x{Z2,  Z3,...,  Zk+ 2}] 

(26) 

where  k  =  0. . . . ,  2L  -  3.  It  is  straightforward  to  check 
that  for  any  k  =  0, . .  M  2L  —  4,  we  have  Pcse  <  Pcs*1^ 
Indeed,  including  the  extra  element  Zfc+3  to  { Z2 , . . . ,  Zk+ 2} 
can  only  increase  the  probability  that  the  maximum  element 
of  {Z2,..., Z*+2,Z*+3}  will  be  larger  than  max{Z0,Zi}. 
Sorting  the  elements  by  decreasing  variance  implies  that  the  k 
elements  with  the  highest  variance  (and  mean)  will  produce  the 
tightest  lower  bound  Pcse  of  all  other  choices  of  k  elements 
from  {Z2,...,Zfc+2}.  For  k  =  2L  -  3,  all  variables  Zj, 
j  =  0, . . . ,  2L  -  1,  are  included  in  the  evaluation  of  the  kth 
bound,  so 

rg?-*>  =  1 

-Pr[max{Zo,Zi}  >  max{Zi,Z2,...,Z2i,-i}].  (27) 

JThe  cardinality  is  one  if  and  only  if  r0  =  nTc  for  some  n  =  0, . . . ,  L  —  1 . 
However,  the  probability  of  these  events  is  zero. 

2Sorting  the  variables  Zj  in  terms  of  decreasing  variance  is  equivalent  to 
sorting  them  in  terms  of  decreasing  mean. 
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Thus  [cf.  (14)] 

p(0)  <P(l)  ^  ^  p(2L— 3) 

=1  -  Pr  [max  {  Zj\j  =  0, . . .  ,21  -  1,  j  g  H(t0)} 

-  max  {  Zk\ k  €  H(tq)}  <  0] .  (28) 

The  probability  bounds  Pete,  k  =  0, ...  ,2L  -  3,  can  be 
evaluated  as  follows.  The  pdf  of  max{Zo,Zi}  is 

/max{2o.^i}(z)  =  fo(z)Fi(z)  +  fl(z)Fo(z)  (29) 
while  the  pdf  of  max{Z2,  •  •  • ,  Zk+ 2}  is 

k+2  /  k+2  \ 

/raax{Z2 . Zk+-2}(z)  =  I  fl(z)  JI  Fj(z )  J  .  (30) 

t=2  \  J=2.  j#l  / 

Then,  the  probability  Pc(se  is  equal  to 

/O  r+oc 

/  fmtLx{Z2,...,Zk+7}(z  “  V ) 

■oo  J— oc 

'Xfmax{Zo,Zl){-y)dydz.  (31) 

Alternatively,  a  simpler  recursive  procedure  for  the  evaluation 
of  the  probability  bounds  Pete ,  k  =  0, . . . ,  2L  -  3,  is  proposed 
below.  The  probability  P^tt,  k  =  0, . . . ,  2L  -  3,  is  equal  to 

where  =  1“P  -  4°i}  <  o]  (32) 

,  Zftax  =  ma x{Z2 , . . . ,  Z*+2  }  (33) 

and 

zmai]  =  rnax{Z0,  Zx}.  (34) 

Assuming  that  Zmax  and  are  normally  distributed,  we 

may  calculate  P^te  as  follows: 

...  (  p{z^x}-p{zffi}}  \ 

*  g  -  1  l  1  l  .  (35) 

y  Var  {z^x}  +  Var  {z,{^}}  j 

The  mean  and  variance  of  Zmax  can  be  evaluated  recursively  as 
described  by  the  following  lemma. 

Lemma  2:  For  k  =  1, . . . ,  2L  -  3 

'  E\z{rt^)}-E{Zk+2}  ^ 

x  G  *•  l 

^  yVar  ^Zrnax  +  Var  {Zk+2}  j 
+  E{Zk+ 2} 

(  E  {Zmax^}  -  E  {Zk+2}  ) 

X  G  — 7-  1  = 

^  y  Var  |Zmax^|  +  Var  {Zk+2}  j 
+  ^Var  {z^}  +  Var  {Zk+2} 

(  E{z^1)\-E{Zk+2}  \ 

x/  «  I  _ t _ J  I  /ItlX 


x  9  I  -•/  \  . .  (36) 

J  Var  {Z^ax15 }  +  Var  {Zk+2}  \ 


Var{z(11lx}4Var{z(f-1)} 

(  E{z^)\-E{Zk+2}  > 

x  G  ^  -  ■  . 

^yVar{z^1)}+Var{Zfc+2}; 

+  Var  {Zj.+2} 

f  E{z^1)}-E{Zk+2}  ^ 
x  G  — j==  === 

^  y  Var  |Zmax1)}  +  Var  {Zjt+2}  j 

(  E  {Z^l)}  -  E  {Zk+2}  > 

G  x  L  . = 

Var  {Zmax1)}  +  Var  {Zk+2}  j 
+  ^Var  +  Var  {Z*+2} 

f  p{z^1)|-P{zfc+2}  \l 

Xfl  -  i  l 

\  y  Var  { ^™ax  ^ ]  4"  Var  { Zfc+2}  j  ■ 

x  (^{z^J-PiZ^}) 

f  E{z^1)]-E{Zk+2}  ) 
^  y  Var  ^ar  {^*+2} y 

-  ^Var  {Z^1}}  +  Var  {Zk+2} 

(  E{z^)}-E{Zk+2}  \] 

x  g  —z  K  J 

{  y  Var  {Zmaxl)}  +  Var  {Zjt+2} 

(37) 

with  E  {z£2x}  =  E{Z2}  and  Var  {zL°ix}  =  Var{Z2}. 
Proof:  Since 

Zmic  =  max  |z2). . . ,  Zk+2}  =  maxfZ^-1),  Zfe+2}  (38) 

(36)  and  (37)  provide,  respectively,  the  mean  and  variance  of  the 
maximum  of  two  uncorrelated  Gaussian  random  variables  [18]. 

■ 

The  mean  and  variance  of  zi°^  can  be  evaluated  in  exactly 
the  same  way.  As  a  final  comment,  we  recall  that  (24)  was  de¬ 
rived  based  on  the  assumption  that  the  pdf  of  the  received  vec¬ 
tors  can  be  closely  approximated  by  the  pdf  of  i.i.d.  Gaussian 
random  vectors.  The  recursive  method  for  the  evaluation  of  Pcse 
developed  in  this  section  is  based  on  the  additional  assumption 
that  the  maximum  of  two  independent  Gaussian  random  vari¬ 
ables  is  approximately  Gaussian.  The  effect  of  both  assump¬ 
tions  is  examined  in  the  simulations  section  through  compar¬ 
isons  with  the  exact  Pcse. 


Qx  - 
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TABLE  III 

Definition  of  C(N) 


V.  Discussion 

Summarizing  the  developments  so  far,  the  probability  of 
coarse  synchronization  error  was  approximated  as  follows: 

Pcse  ~  1  -  Pr[arg  max{  Zj  \  j  =  0, . . . ,  2L  -  1}  €  ft(r0)] 

(39) 

where  ft  (to)  is  the  set  that  contains  the  indexes  of  the  filters 
that  yield  a  timing  estimate  within  the  pull-in  range.  The 
random  variables  Zo,  . . . ,  Z2L-1  are  uncorrelated  Gaussian 
random  variables  with  means  and  variances  given  by 

E{Zj}  =  U)'\  Var{Zj}  =  C(N)U]/3  (40) 

where  the  quantities  Uj,  j  =  0, . . . ,  2L  -  1,  and  the  functions 
C(N )  are  presented  in  Tables  I  and  HI,  respectively,  for  each 
filter  under  consideration  (SMI-MVDR  or  MF  of  order  L  or 
2  L). 

Thus,  the  probability  of  coarse  synchronization  error  was  cal¬ 
culated  as  the  probability  that  the  index  of  the  largest  Zj,  j  — 
0, . . . ,  2 L  -  1,  is  not  contained  in  ft(ro).  This  approach  led  to 
the  Pcse  expressions  for  the  order-L  and  order-2 L  MF-type  or 
SMI-MVDR-type  receivers  presented  in  Section  III,  as  well  as 
to  the  evaluation  of  Pcse  through  the  sequence  of  lower  bounds 
presented  in  Section  IV.  An  intuitively  simpler  but  computation¬ 
ally  more  complex  method  for  the  calculation  of  Pcse  is  pre¬ 
sented  in  the  Appendix.  Expression  (40)  and  Table  III  imply 
that  the  mean  and  variance  of  Zj,  j  =  0, . . . ,  2L  -  1,  and,  con¬ 
sequently,  the  performance  of  the  SMI-MVDR-type  algorithms 
depends  explicitly  on  the  filter  order.  Moreover,  the  data  record 
size  N  and  the  filter  order  appear  only  in  the  terms  ( N  —  2 L) 
and  ( N  -  L )  of  C(N)  for  the  filters  of  order  2 L  and  P,  re¬ 
spectively.  Although  the  filter  order  does  not  affect  the  asymp¬ 
totic  (as  N  —>  oc)  performance  of  the  SMI-MVDR-type  al¬ 
gorithms,  it  does  cause  a  performance  drop  in  the  small  data- 


record-size  operating  region,  since  the  filter-order  term  is  sub¬ 
tracted  from  the  data-record-size  term.  In  other  words,  the  data- 
record  size  is  effectively  reduced  by  a  number  of  data  vector 
samples  equal  to  the  filter  order.  The  longer  the  employed  filter 
is,  the  greater  the  effective  data-record-size  reduction  is.  For  the 
SMI-MVDR-type  coarse  synchronization  algorithms,  this  trans¬ 
lates  to  poorer  performance  under  short  data-support  conditions 
as  the  filter  order  increases. 

An  alternative  way  to  quantify  the  effect  of  the  data-record 
size  on  the  probability  of  coarse  synchronization  error  is  to  ex¬ 
amine  the  short  data-record  synchronization  resolution  of  the 
SMI-MVDR-type  algorithms  that  we  define  as  follows.  For  the 
pair  of  decision  variables  ( Zi ,  Zm ),  l  =  0, . . . ,  2L  -  1,  m  = 
0, . . , ,  2L  -  1,  with  E{Zi}  ^  E{Zm },  the  resolution  metric  is 
defined  by 

4  (Pi[Zi>Zm],  ifE{Zl}>E{Zm} 

Pl,m  \  Pt[Zi  <  Zm],  otherwise.  ^ 

That  is,  the  resolution  pi)Tn  is  the  probability  that  the  coarse  syn¬ 
chronization  algorithm  will  decide  in  favor  of  the  asymptotically 
largest  decision  variable.  In  other  words,  the  synchronization 
resolution  is  a  measure  of  how  close  the  performance  of  the  fi¬ 
nite  data-record  SMI-based  algorithm  is  to  the  asymptotic  per¬ 
formance.  It  is  straightforward  to  show  that  the  resolution  of  the 
SMI-MVDR  algorithms  is  given  by 

(  \ul/z-ull3  \ 

\c{N)-x/2-j==^j  (42) 

where  Q(x)  is  the  cdf  of  a  normal  random  variable  with  mean 
zero  and  variance  one.  The  following  proposition  compares  the 
resolution  performance  of  the  Z-order  and  2L-order  algorithms 
for  a  given  pair  of  hypotheses.  The  proof  is  straightforward  and 
thus  omitted. 

Proposition  1:  For  a  given  pair  of  hypotheses  (Z,m),  l  = 
0, . . . ,  2L  -  1,  m  =  0, . . . .  2L  -  1,  we  have 


if  and  only  if 


9(jy  —  2Z>)  —  2  y/iv^T 
9 (N  -  L)  -  2  s/N=2l  -  S,'m  (44) 

where  5(,m  is  given  by  (45)  as  shown  at  the  bottom  of  the  page. 


Let  5min  —  min{5;,m|Z,m  =  0,  ...,2L-  1}.  Then,  the 
resolution  performance  of  the  L-order  algorithm  will  be  better 
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than  the  performance  of  the  2£-order  algorithm  (in  the  sense 
that  the  L-order  algorithm  performance  will  be  closer  to  the  cor¬ 
responding  asymptotic  performance)  for  any  pair  of  hypotheses, 
provided  that 

9 {N  -  2L)  -  2  v/aTTL 

9(TV  -  L)  -  2  JN-2L  ~  5min'  (46) 

Equation  (46)  identifies  a  condition  on  the  system  parameters 
and  the  data-record  size  under  which  SMI-MVDR-type  L-order 
algorithms  outperform  the  21-order  algorithms. 

On  the  other  hand,  in  the  case  of  MF-type  algorithms,  the 
mean  and  variance  of  Zj  do  not  depend  explicitly  on  the  filter 
order.  Theoretically,  this  conclusion  is  a  direct  consequence 
of  the  fact  that  for  any  Wishart  Wp(E;TV)  matrix  X  and  any 
nonzero  vector  v  €  W,  the  quantity  vTXv  is  a  x2  random 
variable  that  maintains  TV  DOFs  (while  (vTX-1v)-1  is  x2 
with  TV  -  p  DOFs).  Therefore,  algorithms  that  do  not  require 
the  inversion  of  a  Wishart  matrix  do  not  suffer  from  effective 
data-record-size  reduction.  Consequently,  it  is  expected  that  the 
MF-type  and  AV-type  coarse  synchronization  algorithms  are 
fairly  insensitive  to  the  choice  of  filter  length. 

As  a  final  note,  we  recall  that  an  alternative  to  direct  inversion 
of  ^  is  the  use  of  a  matrix-inversion  lemma-based  recursive 
least-squares  (RLS)  algorithm  [16].  We  can  show  that  the  RLS 
algorithm  computes  the  inverse  of  a  diagonally  loaded  version 
ofR(,),  (R(,)  +  (e/TV)I)-1,  where  (l/e)I  is  the  initialization 
matrix  of  RLS.  Therefore,  although  RLS  provides  an  asymp¬ 
totically  (TV  ->  oo)  unbiased  estimate  of  (RW)-1,  for  a  finite 
data-record  size  TV  it  provides  a  biased  estimate  of  (R((>)-i 
with  a  bias  that  depends  on  e/TV.  Since  in  practice,  e  is  small, 
the  effect  of  the  diagonal  loading  quickly  becomes  negligible 
and,  consequently,  the  performance  is  close  to  the  SMI  perfor¬ 
mance  analyzed  in  this  paper. 

VI.  Simulation  Results  and  Comparisons 

We  consider  a  10-user  asynchronous  DS-CDMA  system  that 
utilizes  Gold  sequences  of  length  L  =  31.  We  compare  the 
CSER  evaluated  using  the  expressions  derived  in  Sections  III 
and  IV  with  the  exact  CSER  of  MF  and  SMI-MVDR-type  algo¬ 
rithms.  The  exact  CSER  is  evaluated  by  averaging  over  10000 
independent  runs.  The  delays  of  all  users  are  chosen  randomly 
and  kept  constant  for  the  duration  of  the  experiment. 

In  Figs.  3  and  4,  the  SNR1  of  the  user  of  interest  is  10  dB 
and  the  data  support  size  ranges  from  35  to  80  samples.  The 
SNRs  of  the  interferes  are  fixed  at  6,  8,  10,  12,  15,  16,  18,  20, 
and  22  dB.  In  Fig.  3,  we  plot  as  a  function  of  k  the  sequence  of 
bounds  PCse  developed  in  Section  IV  for  the  order  L  and  order 
2 L  SMI-MVDR-type  algorithms  [given  by  (35H37)].  In  Fig.  4, 
we  compare  the  exact  coarse  synchronization  performance  of 
order-L  and  order- 2  L  algorithms  to  the  derived  approximations 
of  Sections  III  and  IV,  that  is,  the  direct  approximation  through 
(24)  and  the  recursive  approximation  through  (35)— (37).  Re¬ 
gardless  of  the  filter  order,  we  see  that  both  approximations  offer 
close  coarse  synchronization  performance  estimates.  We  ob¬ 
serve  that  the  SMI-MVDR  L-order  algorithm  exhibits  improved 

3The  SNR  of  user  k,  k  =  0 . A*  -  1.  is  defined  as  Ek/a2,  where  a2  is 

the  variance  of  the  chip-matched  filtered  and  sampled  AWGN. 


Fig.  3.  Recursive  approximation  of  the  CSER  of  SMI-MVDR-type  receivers 
of  order  L  and  order  21  given  by  (35H37)  (SNR0  =  10  dB). 


Data  record  size  N 


Fig.  4.  CSER  of  linear  receivers  of  order  L  and  21  as  a  function  of  the 
data-record  size  N  (SXR0  =  10  dB). 

performance  when  compared  with  the  SMI-MVDR  2L-order 
scheme  for  data-record  sizes  above  L  =  31.  In  fact,  in  the  small 
data  support  region,  the  L-order  algorithm  can  achieve  the  same 
CSER  as  the  2L-order  algorithm,  about  L  samples  faster. 

In  Figs.  5  and  6,  we  repeat  the  studies  of  Figs.  3  and  4  for  dif¬ 
ferent  values  of  the  SNR  of  the  user  of  interest.  The  data  support 
for  the  estimation  of  the  order- L  and  order-2L  SMI-MVDR  fil¬ 
ters  is  TV  =  55  and  80  samples,  respectively.  The  results  parallel 
the  findings  in  Figs.  3  and  4. 

The  insensitivity  of  the  noninversion-based  coarse  synchro¬ 
nization  algorithms  to  the  filter  order  is  illustrated  by  the  simu¬ 
lation  study  of  Fig.  7,  where  the  probability  of  coarse  synchro¬ 
nization  error  of  the  L-order  algorithms  is  plotted  as  a  function 
of  the  system  processing  gain  L  (the  figure  also  includes  the 
AV-type  coarse  synchronization  scheme  presented  in  [7]).  The 
user  SNR  values  are  identical  to  those  in  Fig.  3  and  the  data 
support  for  filter  estimation  is  TV  =  125  samples.  The  signa¬ 
ture  vectors  of  the  users  are  constructed  by  concatenating  Gold 
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_  _  .  ,  ____  r  mu  i«rnn  •  Fig.  7.  CSER  of  linear  receivers  of  order  L  as  a  function  of  the  processing 

Fig.  5.  Recursive  approximation  of  the  CSER  of  SMI-MVDR-type  receivers  ?  j 

of  order  L  and  order  2X  given  by  (35M37)  (Ar  =  55  and  80,  respectively).  £am 


Order-L  (N=55) 


SNR  of  user  0 
Order-2L  (N=80) 


SNR  of  user  0 

Fig.  6.  CSER  of  linear  receivers  of  order  L  and  2  L  as  a  function  of  the  SNR 
of  the  user  of  interest  (AT  =  55  and  80,  respectively). 


a  double  integral.  The  second  approximation  method,  which 
is  the  least  computationally  complex,  provides  a  sequence  of 
increasingly  tight  lower  bounds  on  the  approximate  expressions 
of  the  first  method.  Finally,  the  third  and  most  intuitive  method 
approximates  the  probability  of  coarse  synchronization  error  as 
a  2L-order  integral  (the  expression  is  given  in  Appendix).  The 
analytical  expressions  provide  simple,  highly  accurate  alterna¬ 
tives  to  computationally  demanding  performance  evaluation 
through  simulations.  We  showed  that  the  coarse  synchroniza-  - 
tion  performance  of  the  MF-type  receiver  is  a  function  of  the 
data-record  size,  while  the  performance  of  MVDR-type  SMI 
or  RLS-based  filter  estimators  is  a  function  of  the  difference 
between  the  data-record  size  and  the  employed  filter  order. 
The  latter  translates  to  an  effective  reduction  of  the  data-record 
size  with  a  more  evident  negative  effect  on  the  performance 
as  the  available  data  record  becomes  smaller.  MF-type  and 
AV-type  (which  provide  active  interference  suppression)  filter 
estimators  do  not  suffer  such  data  reduction  and  thus,  provide 
an  attractive  solution  when  the  environment  dictates  short 
data-record  synchronization. 


codes  of  length  3 1 .  We  observe  that  the  SMI-MVDR  algorithm  APPENDIX  I 

is  severely  affected  by  the  value  of  the  spreading  gain  (filter 

length)  as  L  approaches  N ,  while  the  MF-type  algorithm  is  not  A.  Proof  of  Lemma  1 

affected  (but  has  unacceptable  synchronization  performance,  as  ^  marginal  0f  Ujt  j  =  0, . . . ,  2L  -  1,  is 
also  seen  in  Figs.  4  and  6).  The  AV-type  synchronizer  does  not 

suffer  from  significant  performance  degradation  as  the  system  xjv/2-i  exp  f  -u\ 

spreading  gain  L  increases,  and  offers  by  far  the  most  reliable  hAu)  = - - —  •*  . 

synchronization.  (^i)  r(f) 


(47) 


VII.  Conclusions 

In  this  paper,  we  investigated  the  finite  data-record  coarse 
synchronization  performance  of  blind  adaptive  combined  syn¬ 
chronizers/demodulators.  Using  transformation  noise  modeling 
techniques,  we  derived  analytical  expressions  that  approxi¬ 
mate  closely  the  probability  of  coarse  synchronization  error. 
Three  alternative  methods  were  developed.  The  first  method 
approximates  the  probability  of  coarse  synchronization  error  as 


Expression  (47)  implies  that  the  random  variable  Uj, 
j  —  0, . . .  ,2L  -  1,  is  a  multiple  of  a  chi-square  (x2) 
random  variable.  More  precisely,  Uj  can  be  written  as 

Uj  =  (Lu^j  Xj,  j  =  0, . . . ,  2L  -  1  (48) 

where  Xj  is  a  x2  random  variable  with  N  DOFs. 
Since  ( Xj/N )1/3  is  approximately  Gaussian  with  mean 
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(1  -  (2/9 N))  and  variance  2/9 N  [19],  we  may  approximate 
the  pdf  of  Uj  by  the  pdf  of 

V^N^U^Y^UjY?,  j  =  0,...,2L-1.  (49) 

In  the  above  equation,  Yj  is  a  Gaussian  variable  with 
mean  (1  —  (2/9iV))  and  variance  2/9 N.  The  joint  pdf  of 
Yo,  •  • .  will  be  completely  defined  if  we  specify  their 

covariance  matrix  that  we  denote  by  S'.  The  diagonal  elements 
of  S'  are  equal  to  2/9AT.  The  nondiagonal  elements  of  S'  are 
chosen  such  that  Vb, . . . ,  V2l-i  have  covariances  identical  to 

the  covariances  of  the  corresponding  C7b, _ ,  #21-1  variables. 

The  latter  covariances  are  [20] 

Cov  [tfj.tf*]  =  ^  (dyoii  .  (50) 


Fig.  8.  Positive  root  of  (53)  as  a  function  of  a  and  data-record  size  .V. 


To  find  the  covariances  ipjk  of  Yj  and  Yk,  j  ^  k,  that  result  in 
Cov  [Vj,  14]  =  —  (dpu  Rd||0j|]  (51) 


Appendix  II 

A.  An  Alternative  Approximation  ofPcse 

The  probability  of  coarse  synchronization  error  can  be  ap¬ 
proximated  by  [cf.  (14)] 


we  substitute  (49)  in  (51)  and  make  use  of  the  moment  theorem 
to  evaluate  E  {Y^yfc3}.  Then,  we  find  that  the  (j,  fc)th  nondiag¬ 
onal  element  of  'J',  ipjk,  must  satisfy 


2  1 

(du)T 

2 

1 

N{ 

nioii  itanon  j 

II 

fd(k)T 

(52) 


or,  equivalently 


(4-18N-81N2)2  ,  1 

+  6  x  6561AT4  ^j*_3ArQ~°  (53) 


where 


Pcse  ~  1  -  Fv[a,vgmax{ Zj\j  =  0, . . .  ,2L  -  1}  € 

(54) 

where  W(r0)  is  the  set  that  contains  the  indexes  of  the  filters 
that  yield  a  timing  estimate  within  the  pull-in  range.  The  random 
variables  Zj,  j  =  0, . . . ,  2L  - 1,  are  independent  Gaussian  with 
mean  Uj^3  and  variance  C{N)U^3,  where  the  function  C(N) 
is  given  in  Table  III  for  each  of  the  receivers  under  consideration. 
The  pdf  of  the  random  vector  Z  =  [Z0, . . . ,  Z2l- i]t  is 


/z(z)  = 


(2»)rnS  ly[c{N)ufl 


-J  y'kiULl 

<  h  2 C(N)U]/3 


(55) 


where  z  =  [z0. . . . ,  z2i-i]t.  Thus 


where 


(56) 


By  the  Schwarz  inequality  a  €  (0, 1).  Equation  (53)  has  three 
real  roots,  two  of  which  are  always  negative,  provided  that  N  > 
4.  The  third  root  is  plotted  in  Fig.  8  as  a  function  of  N  and  a.  We 
see  that  the  third  root  is  always  positive  while  it  drops  rapidly 
to  zero  as  N  increases.  In  fact,  for  N  >  40,  we  see  that  tpjk  is 
less  than  0.0045.  Thus,  we  may  assume  that  ipjk  =  0,  j  ^  jfc, 
j,  k  =  0, . . . ,  2L  —  1.  In  other  words,  we  can  safely  assume 
that  the  Gaussian  random  variables  Yj,  j  =  0, . . . ,  2L  -  1,  are 
uncorrelated  and,  therefore,  independent.  ■ 


=  U  ]  (do»di ?*•••> ^2L-i)  €  3?2L  with 

ken(ro)  ^ 

dk  >  dj,j  =  0,...,2L-l,j^  k\.  (57) 
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Abstract 


We  investigate  the  data  record  size  requirements  of  sample-matrix-inversion-based 
minimum-variance-distortionless-response  and  maximum  signal-to- interference-plus- 
noise-ratio  adaptive  algorithms  to  meet  a  given  performance  objective  in  joint  space- 
time  signed  detection  problems  for  direct-sequence  code-division-multiple-access  sys¬ 
tems.  We  derive  closed  form  expressions  that  provide  the  data  record  size  that  is 
necessary  to  achieve  a  given  performance  confidence  level  in  a  neighborhood  of  the 
optimal  performance  point  as  well  as  expressions  that  identify  the  performance  level 
that  can  be  reached  for  a  given  data  record  size.  This  is  done  by  utilizing  close  approx¬ 
imations  of  the  involved  probability  density  functions.  The  practical  significance  of  the 
derived  expressions  lies  in  the  fact  that  the  expressions  are  functions  of  the  number 
of  antenna  elements,  the  number  of  multipaths,  and  the  system  spreading  gain  only, 
while  they  depend  neither  on  the  ideal  input  covariance  matrix  which  is  not  known  in 
most  realistic  applications  nor  on  the  exact  ideal  performance  value. 


Keywords:  Antenna  arrays,  code  division  multiaccess,  direction  of  arrival  esti¬ 
mation,  finite  data  analysis,  interference  suppression. 
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I.  Introduction 


In  this  paper  we  focus  on  direct-sequence  code-division-multiple-access  (DS/CDMA)  single- 
user  detection.  We  consider  multipath  fading  additive  white  Gaussian  noise  (AWGN)  chan¬ 
nels  and  antenna  array  signal  reception,  either  at  the  base  station  or  at  the  mobile  end 
(for  example  airborne  arrays)1.  We  recall  that  single-user  DS/CDMA  detection  aims  at  the 
recovery  of  the  information  bits  of  one  user  that  we  call  “the  user  of  interest.”  The  only 
quantity  assumed  known  is  the  effective  (space-time  channel  processed)  signature  of  the  user 
of  interest  which  is  a  function  of  the  multipath  fading  coefficients,  delays,  and  multipath 
directions  of  arrival  (the  interfering  signal  statistics  are  assumed  unknown). 

We  consider  a  receiver  that  consists  of  either  the  minimum- variance-distortionless-response 
(MVDR)  or  the  maximum-signal-to-interference-plus-noise  ratio  (MSINR)  linear  filter  fol¬ 
lowed  by  a  sign  detector.  The  MVDR  linear  filter  is  found  by  minimizing  the  filter  output 
variance  subject  to  the  constraint  that  the  filter  remains  distortionless  in  the  effective  sig¬ 
nature  vector  direction  of  the  user  of  interest  [1],  [2].  The  filter  is  a  function  of  the  inverse 
input  autocovariance  matrix  and  the  effective  signature  of  the  user  of  interest.  On  the  other 
hand,  the  MSINR  filter  is  found  by  maximizing  the  signal-to-interference-plus-noise  ratio  at 
the  linear  filter  output  and  it  is  a  function  of  the  inverse  interference-plus-noise-only  autoco¬ 
variance  matrix  and  the  effective  signature  of  the  user  of  interest.  The  MVDR  and  MSINR 
linear  filters  are  scaled  versions  of  each  other  and,  thus,  under  perfectly  known  statistics  they 
exhibit  identical  performance  in  terms  of  both  filter  output  SINR  and  receiver  bit-error-rate 
(BER). 

The  sample-matrix-inversion  (SMI)  estimate  of  the  MVDR  or  the  MSINR  filter  is  ob¬ 
tained  by  substituting,  respectively,  the  inverse  of  the  ideal  autocovariance  matrix  of  the 
input  or  the  interference-plus-noise  part  only  by  the  inverse  of  the  corresponding  sample 

average  estimate.  The  corresponding  filter  estimate  is  called  SMI-MVDR  or  SMI-MSINR 

'Single-antenna  signal  detection  systems  constitute  a  special  case  of  the  general  system  that  we  consider 
here. 
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filter.  These  estimates  are  occasionally  referred  to  in  the  literature  as  SMI- “signal-present”  or 
SMI- “signal-absent”  filter,  respectively.  Under  asymptotically  many  data,  the  SMI-MVDR 
filter  and  the  SMI-MSINR  filter  exhibit  identical  performance  in  terms  of  both  filter  output 
SINR  and  receiver  BER  since  they  converge  in  probability,  under  general  conditions,  to  their 
ideal  counterparts.  This  is  not,  however,  the  case  when  estimation  with  finite  data  records  is 
performed  where  for  small  data  record  sizes  the  SMI-MSINR  outperforms  the  SMI-MVDR 
filter  in  terms  of  output  SINR.  Along  these  lines,  we  note  that  the  sample-average  estimate 
of  the  interference-plus-noise  autocovariance  matrix  can  be  evaluated  from  data  obtained 
either  by  probing  the  channel  prior  to  the  beginning  of  the  desired  transmission  (or  during 
the  silent  periods  of  the  user  of  interest)  or  by  estimating  and  then  subtracting  the  desired 
user  signal  from  the  received  signal.  In  multipath  environments  the  interference  component 
in  the  former,  desired-signal-absent,  case  does  not  contain  inter-symbol-interference  (ISI) 
and  thus  is  not  exactly  the  same  as  the  interference  component  in  the  latter,  “estimate  and 
subtract”,  case  (which  contains  ISI).  However,  in  spread-spectrum  systems  the  channel  delay 
spread  is  much  less  than  the  information  symbol  period  and  we  can  safely  ignore  the  effect 
of  ISI  and  treat  all  cases  in  the  same  way. 

The  goal  of  this  paper  is  twofold.  We  would  like  to  identify  the  data  record  size  that  is 
necessary  for  (a)  the  SMI-MVDR  DS/CDMA  multiple-access-interference  (MAI)  suppression 
filter,  and  (b)  the  SMI-MSINR  DS/CDMA  MAI  suppression  filter  to  achieve  a  given  perfor¬ 
mance  confidence  level  in  a  neighborhood  of  the  optimal  performance  point.  Performance 
is  measured  in  terms  of  filter  output  variance  and  filter  output  SINR,  respectively.  We  also 
investigate  the  dual  problem  of  quantifying  the  relative  performance  of  the  above  estimators 
with  respect  to  the  performance  of  the  corresponding  ideal  schemes  for  a  given  data  record 
size.  In  pursue  of  these  objectives  we  define  the  following  relative  performance  measures  for 
the  estimated  filters  with  respect  to  their  ideal  counterparts  in  one-to-one  correspondence 
with  (a)-(b)  above:  (a)  The  ratio  of  the  output  variance  of  the  ideal  filter  over  the  output 
variance  of  the  estimated  filter,  and  (b)  the  ratio  of  the  output  SINR  of  the  estimated  filter 
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over  the  output  SINR  of  the  ideal  filter.  Our  approach  is  based  on  the  observation  that  a 
near  optimum  performance  level  is  reached  by  any  estimation  scheme  when  the  correspond¬ 
ing  ratio  is  close  to  unity.  Thus,  any  given  ( or  desired)  confidence  on  the  performance  level 
can  be  expressed  as  the  probability  that  the  above  measures  are  within  a  certain  neighbor¬ 
hood  of  unity.  This  approach  is  in  contrast  to  the  work  in  [3]  where  the  measures  (a)  and  (b) 
were  used  to  determine  the  data  record  that  the  estimated  filter  needs  to  achieve  on  average 
a  given  percentage  of  the  optimum  performance.  In  addition,  in  this  paper,  making  use  of 
the  functional  relationship  between  the  BER  and  the  SINR  of  a  linear  receiver,  we  extend 
our  developments  summarized  above  to  evaluate  our  confidence  on  the  BER  performance 
of  DS/CDMA  single-user  detectors  that  utilize  the  SMI-MVDR  or  SMI-MSINR  filter,  re¬ 
spectively.  The  practical  significance  of  the  theoretical  developments  in  this  paper  lies  in 
the  fact  that  by  utilizing  simple  yet  close  approximations  of  the  involved  probability  density 
functions  we  are  able  to  derive  closed  form  expressions  that  are  functions  of  the  number 
of  antenna  elements,  the  number  of  multipaths,  and  the  system  processing  gain  and,  most 
importantly,  independent  of  the  ideal  input  statistics. 

The  paper  is  organized  as  follows.  In  Section  II  we  present  the  system  model.  In  Section 
III  we  outline  the  MVDR  and  MSINR  joint  space-time  processors,  we  analyze  the  perfor¬ 
mance  of  their  SMI  estimators  as  a  function  of  the  data  record  size,  and  we  derive  their 
data-record-size  requirements  to  achieve  a  given  performance  level.  A  few  conclusions  are 
drawn  in  Section  IV. 


II.  System  Model 

We  consider  a  DS/CDMA  system  with  K  users  transmitting  over  a  multipath  fading  additive 
white  Gaussian  noise  (AWGN)  channel.  The  information  bits  of  each  user  are  binary- phase- 
shift-keying  (BPSK)  modulated.  The  kth  user  baseband  transmitted  signal  is  given  by 

Mt)  =  Y.bk^^Sk(t-iT),  k  =  0,...,K  —  1,  (1) 
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where  bk(i)  G  {-1, 1}  is  the  ith  data  (information)  bit  and  T  is  the  information  bit  period. 

Ek  represents  transmitted  energy  per  bit  and  Sk(t)  is  the  signature  waveform,  assumed  to 

be  normalized  to  unit  energy  per  data  bit  period,  given  by 

Sk(t)  =  J2sk(im-lTc)  (2) 

/=0 

where  sk(l)  G  {-1, 1}  is  the  lih  bit  of  the  spreading  sequence  of  the  kth  user,  Tc  is  the  chip 
period,  ip(t)  is  the  chip  waveform  (assumed  to  be  a  rectangular  pulse  with  support  [0,TC]), 
and  L  =  T/Tc  is  the  system  spreading  gain.  The  fcth  user  signal  uk(t)  is  transmitted  over  a 
multipath  fading  channel  with  impulse  response 

Np- 1 

hk(t)  —  ^  ^  Ck,n5{t  ~  Tk,n)  (^) 

71=0 

where  Np  is  the  number  of  paths  (assumed  the  same  for  all  users),  ckt„,  k  =  0, . . .  ,K  -  1, 
n  =  0, . . . ,  Np  —  1  are  the  complex  path  coefficients,  and  6(t)  is  the  Dirac  delta  function.  The 
coefficients  are  assumed  to  be  identical  across  all  antenna  elements,  that  is,  antenna  diversity 
effects  are  not  considered.  Finally,  rfc>n  is  the  relative  transmission  delay  of  the  nth  path  of 
the  Arth  user  signal.  For  simplicity  in  presentation,  we  assume  that  the  receiver  is  coarsely 
synchronized  with  the  first  path  of  the  user  of  interest,  user  0,  that  the  channel  delay  spread 
is  equal  to  Np  chips  and  that  no  resolvable  path  delays  fall  within  the  same  chip  period,  i.e. 

T k,n  —  Vk,n  +  <5fc,n>  A  =  0, . . . ,  K  —  1,  n  =  0, . . . ,  Np  —  1,  (4) 

where  uk>n  is  an  integer  between  0  and  L  —  1  (with  I'o.n  =  n)  and  SktTl  lies  in  [0, 1). 

The  received  signal  is  collected  by  a  uniform  linear  antenna  array  consisting  of  M  elements 
spaced  half-the- wavelength  apart.  The  baseband  received  signal  at  the  mth  antenna  element 
is  given  by 

K- lNp-l 

rM  =  E  E  ck,nUk(t  -  Tk,n)e-jmMk'n  +  nm(t),  m  =  0, . . . ,  M  -  1,  (5) 

k= 0  n= 0 

where  9kjn  identifies  the  angle  of  arrival  of  the  nth  path  of  the  kth  user  signal  and  nm(t) 
represents  additive  sensor  noise  modeled  as  temporally  and  spatially  complex  white  Gaussian. 
The  received  signals  ro(t), .'. . ,  rM-i(t)  can  be  grouped  to  form  the  vector 

K—l  Np-l 

r(t)  =  [r0{t),ri(t), . . .  ,rM-i(t)]T  =^2  c^uk{t  ~  rk,n)a(9k,n)  +  n(t)  (6) 

fc=0  n=0 
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where 


=  k  =  0,...,K  —  1,  n  =  0, . . . ,  jVp  —  1,  (7) 

is  the  steering  vector  associated  with  the  nth  path  of  the  Jfeth  user  and 


n(t)  =  [n0(0,...,nM-iW]r.  (8) 

Chip-matched  filtering  and  sampling  at  the  chip  rate,  1/TC,  of  r(t)  over  the  multipath 
extended  time  interval  (L  +  Np  chip  periods)  prepares  the  data  for  one-shot  detection  of  the 
ith  information  bit  of  interest  b0(i).  By  stacking  the  vector  samples  r(0), . . . ,  r((L+ATp-l)Tc) 
one  below  the  other  we  obtain  the  space-time  received  data  vector 


*M(L+Np)xi  =  [r(0)T  t(Tc)t  . . .  TT({L  +  Np-  1  )Tc)]t  .  (9) 

From  now  on,  r  denotes  the  joint  S-T  data  in  the  Cm(l+np)  complex  vector  domain. 

The  cornerstone  of  joint  space-time  filtering  is  the  space-time  signature  which  for  user  0 
is  defined  as  atp-i 

v0  =  E  co,n  [(1  -  6o,n)sP  +  ^0,«s^+1)l  ®  a(0o>„)  (10) 

M  A  n"°  J 

where  s0  =  [0, . . .  ,0, so(0), . . . , s0(L  -  1),0, ..  . ,0jT  and  ®  denotes  the  Kronecker  product. 

n  ATp-n 

In  the  next  section  we  outline  the  MVDR  and  MSINR  DS/CDMA  joint  space-time  pro¬ 
cessors  and  we  analyze  the  performance  of  the  SMI-MVDR  and  SMI-MSINR  estimators  as 
a  function  of  the  data  record  size.  Also,  we  derive  their  data-record-size  requirements  to 
achieve  a  given  performance  level. 


III.  Data-Record-Size  Requirements  of  Space-Time 
MVDR  and  MSINR  Receivers 

A  linear  joint  S-T  DS/CDMA  interference  suppressing  receiver  with  tap  weight  vector  w  € 
C  (  p)  detects  the  transmitted  bit  of  the  user  of  interest  as  follows: 

b0  =  sgn(Z?e{wHr})  (u) 

where  bQ  denotes  the  detected  information  bit  of  the  user  of  interest,  sgn(-)  identifies  the 
sign  operation,  and  Re{-}  extracts  the  real  part  of  a  complex  number.  In  this  work  we 
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consider  the  MVDR  and  MSINR  linear  receivers.  The  tap-weight  vector  of  the  MVDR  filter 
is  designed  to  minimize  the  filter  output  variance  while  maintaining  unit  response  in  the 
known2  vector  direction  vq  and  equals 


w  MVDR  — 


R~lv0 

Vj?R-1v0 


(12) 


where  R  =  E{rr^}  is  the  covariance  matrix  of  the  received  vector  r  (£'{•}  denotes  the 
expectation  operation).  The  MVDR  filter  output  variance  is 

V(Wm'v'Dr)  =  }  =  ^MVDl (1^) 

The  tap  weight  vector  of  the  MSINR  filter  is  found  by  maximizing  the  SINR  of  the  linear 


filter  output  and  is  given  by 


W  MSINR  — 


R/+nV  0 

V"R7+nV0 


(14) 


where  R/+n  =  R  —  Eov0v<f  is  the  “interference-plus-noise”  covariance  matrix.  The  SINR  at 
the  output  of  the  MSINR  filter  is  given  by 

g(w»M».)  =  waE°|W|S",J9-1— •  <15) 

W  M  SIN  R^I+n™  MSINR 

It  can  be  shown  that  in  ideal  situations  (i.e.,  when  R  and  R /+„  are  perfectly  known) 
wMVDJi  and  yf  msinr  exhibit  identical  performance  in  terms  of  both  output  SINR  and  receiver 


BER.  However,  knowledge  of  the  ideal  covariance  matrices  R  or  R/+n  cannot  be  assumed  in 
practice  and,  thus,  the  matrices  have  to  be  estimated.  We  choose  to  estimate  R  and  R i+n 
by  sample  averaging3.  The  sample  average  estimate  of  R  based  on  a  data  record  of  size  N 
is  given  by  n-i 

<16> 

j — 0 

while  the  sample  average  estimate  of  Rj+n  based  on  a  data  record  of  size  N  is  given  by 

1  tf-1 

R/+»  =  ] v^jJ+nrfl+n  (1?) 

-  _  j=0 

2We  note  that  the  vector  direction  Vo  is  the  only  quantity  assumed  known  (path  coefficients,  path  delays 

and  directions  of  arrival  are  modeled  as  deterministic  unknowns). 

3Other  estimators  of  R  and  R/+n  may  also  be  used  instead  of  the  sample  average  estimators.  For  example, 

at  the  base  station  we  can  estimate  (“reconstruct”)  the  matrices  R  and  R/+n  using  estimates  of  the  energies, 

channel  coefficients,  delays,  and  angles  of  arrival  of  all  users.  However,  the  performance  of  the  corresponding 

MVDR  and  MSINR  filter  estimator  will  strongly  depend  on  the  quality  of  all  intermediate  estimators  used 

and,  thus,  it  will  be  difficult  to  analyze  under  finite  data  record  scenarios. 
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«  > 


where  the  index  (/  +  „)  in  j  =  0 . N  -  1,  identifies  desired-signal-free  received 

vectors.  To  guarantee  that  the  estimates  R  or  R,+„  are  of  full  rank  with  probability  1 
(w.p.  1)  so  that  the  estimates  [fi]  ‘  and  [r,+„]"‘  exist,  the  data  record  size  N  must 
satisfy4  N  >  M(L  +  NP).  We  also  note  that  the  received  vectors  r,  used  for  the  computation 
of  R  are  distributed  according  to  a  mixture  of  up  to  23K  complex  Gaussian  distributions5. 
Unfortunately,  exact  closed  form  expressions  for  the  probability  density  function  (pdf)  of  the 
sample  covariance  matrix  that  is  formed  by  vector  samples  drawn  from  a  Gaussian  mixture 
distribution  are  very  complicated  even  for  the  real  case  [4].  An  additional  complicating  factor 
is  the  underlying  dependence  of  the  vector  samples  r,  due  to  multipath  propagation.  To  that 
extend,  we  make  the  following  simplifying  assumption.  We  assume  that  the  received  vectors 
are  independent  and  identically  distributed  according  to  a  multivariate  complex  Gaussian 
distribution  Af(0,  R).  Then,  the  estimator  R  is  complex  Wishart  distributed  with  N  degrees 
of  freedom,  WM(i,+Arp)(R/7V;  N )  [5].  Similarly,  we  assume  that  R/+n  ^  distributed  according 
to  a  Wishart  distribution  with  N  degrees  of  freedom  of  the  form  Wm(l+wp)(R/+„/./V;  TV). 
Numerical  studies  presented  later  in  this  section  will  demonstrate  the  effect  of  the  above 
assumptions  on  the  developments  of  this  paper. 


The  SMI  estimator  of  the  MVDR  filter  is  obtained  by  substituting  (16)  into  (12),  i.e. 

a  R_1v0 


v  SMI-MVDR  — * 


v^R-Wq' 


The  variance  at  the  SMI-MVDR  filter  output  is 

V(w5M/_WDR)  _  — ■  - (19) 

(v^R-Voj 

We  understand  that  the  variance  V(wsmi_mvdr)  is  lower  bounded  by  V(wMVDR)  =  (v^R_1v0)" 

which  is  the  output  variance  of  the  ideal  MVDR  receiver  in  (12).  This  implies  that  the  vari- 
4For  a  large  class  of  multivariate  elliptically  contoured  input  distributions  that  includes  the  Gaussian,  if 

N  >  M{L  +  Np),  then  R  (or  R/+n)  is  positive  definite  (hence  invertible)  w.p.  1  [6]-[8]. 

5The  total  number  of  Gaussian  distributions  is  equal  to  2*.  where  z  is  the  total  number  of  information 

bits  contributed  by  all  users  to  each  received  vector.  In  our  case;  user  k,  k  =  0, . . . ,  K  -  1,  contributes  to 
the  received  vector  r,  two  bits  (bk(j  -  1)  and  bk(j))  if  min„{r*,n}  >  NPTC  and  three  bits  (bk(j  -  1),  bk(j), 
and  bk(j  +  1)),  otherwise.  Thus,  the  maximum  value  of  x  is  3 K. 
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ance  ratio  py  defined  as 


PV  —  V(wMVI>fj)/V(WsM/--MVDJl)  (20) 

is  upper  bounded  by  unity  (py  <  1).  Near-optimum  performance  of  the  SMI-MVDR  filter 

estimator  is  attained  at  a  value  of  py  close  to  unity. 

Similarly,  the  SMI  estimator  of  the  MSINR  filter  is  obtained  by  substituting  (17)  into 


(14)  and  is  given  by 


*  SMI-MSINR  — 


The  SINR  at  the  SMI-MSINR  filter  output  is 


•RjfinVo 

v*R7+nvo' 


f  SMI-MSINR 


)  a  ^o(v0gR7>o)2 
v?R7+nR/+nR7+nV0 


The  SINR  S('Wsmi-msinr)  is  upper-bounded  by  S(wMSINR)  =  i?o(vo*RJ_Jnvo)  which  is  the 
output  SINR  of  the  ideal  MSINR  filter.  Thus,  near  optimum  performance  of  the  wSMI_MSINR 
filter  requires  that  the  SINR  ratio  ps  defined  as 


Ps  =  S(  w  SMI-MSINR  )/S(  wms/nr)  (23) 

is  close  to  its  maximum  value  of  unity  (ps  <  1). 

Given  the  number  of  antenna  elements  M,  our  goal  is  to  derive  conditions  on  the  data 
record  size  N  necessary  for  the  random  variables  ps  and  py  to  be  sufficiently  close  to  1  in 
the  following  probabilistic  sense: 

Pr[py  >  1  —  £]  >  e  for  any  given  C  €  (0, 1)  and  e  E  (0, 1)  (24) 


and 

Pr[ps  >  1  —  C]  >  c  for  any  given  C  €  (0, 1)  and  e  E  (0, 1).  (25) 

The  performance  measures  in  (24)  and  (25)  assign  a  minimum  confidence  e  to  the  neighbor¬ 
hood  (1— C»  1]  of  the  optimum  performance  point  (py>opt  =  Ps,opt  =  !)•  The  following  theorem 
identifies  the  performance  level  that  can  be  reached  by  the  SMI-MVDR  and  SMI-MSINR 
filter  for  a  given  data  record  size,  as  well  as  the  least  number  of  samples  that  guarantees  (24) 
and  (25). 


Theorem  1  (i)  For  any  given  £  E  (0, 1)  and  N  >  M(L  +  Np)  we  have 
Pr[py  >1  —  (—  ~  +  Np)  +  2) 

V  vW«l  -  0  / 
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and 


paps >i-q« i-Q 2  •  <«) 

(ii)  For  any  given  C  €  (0, 1)  and  e  €  (0, 1),  the  least  number  of  samples  that  guarantees 

Pr[pv  >  1  ~  C]  >  €  or  Pr[/?5  >  1  —  C]  >  c  can  be  approximated  by 

[  max  |m(L  + ATp),  (Wb»+a[M(£+y,)-g]-V7a ■a-0[4M(£+i^)-8+(ro^r  1  ,ye<0' 


Pr[pv  : 

> 1  C] 
?  1 

>  e  or  Pr[ps 

r  r- 

max  \ 

M(i  +  AT,), 

N*  =  i 

max  | 

\m{l  +  JVP), 

(l-C)72+2[M(L4-Np)-2]+v/72(l-0[4M(L+Afp)-8+(l-Ch2l' 

2C 


»/e  <  0.5 


t/e  >  0.5 


where  Q(x)  =  e~v  !2dy,  7  =  Q~l(l  -  e),  [x]  denotes  the  smallest  integer  larger 

than  or  equal  to  x,  and  M,  L  and  Np  denote  the  number  of  antenna  elements,  the  system 
processing  gain ,  and  the  number  of  resolvable  paths,  respectively . 

Proof.  We  can  show  that  />y  and  ps  are  identically  distributed  according  to  a  Beta  distri¬ 
bution  [3]  [11]  with  parameters  N  -  M(L  +  Np)  +  2  and  M(L  +  Np)  -  1.  We  note  that 
the  distribution  of  ps  was  used  in  [12]  to  assess  the  data  record  size  requirements  of  the 
spatial  SMI-MVDR  beamformer  by  visual  inspection  of  the  Beta  function  itself.  In  [3],  the 
distribution  of  ps  was  used  to  evaluate  the  data  record  size  that  is  necessary  for  the  spatial 
SMI-MVDR  beamformer  to  make  the  expected  value  of  p$  greater  than  a  given  constant. 

For  simplicity  in  presentation,  we  denote  comprehensively  both  py  and  ps  by  p.  The 
probability  density  function  (pdf)  of  p  is  given  by 

~~  B  {N  —  M(L  +  iVp)  +  2,  M(L  +  jVp)  -  1}X^  M{i+JVp)+1(l  -  x)M(l+np)-i,  x  e  [^29) 

where  B{-,  •}  is  the  complete  Beta  function6.  Then,  Pr[p  >  1  —  C]  =  /11_c  fp{x)dx.  But  [9] 

/  /»^dx  = 1  -  £  IM1-  o'c"-*.  (30) 

1  C  i-N-M(L+Np)+ 2  \  i  J 

The  right  hand  side  of  (30)  is  the  probability  that  a  binomially  distributed  random  variable 
is  less  than  N  -  M(L  +  Np)  +  2.  Using  the  Gaussian  approximation  [9]  of  the  binomial 
cumulative  distribution  function  we  can  express  (24)  or  (25)  as 

,  f,  ( CN  —  M(L  +  Np)  +  2^  ^  /  x 

_ _ L  Vmi  -  0  )-e  () 

6The  complete  Beta  function  is  defined  as  B(x,y)  =  J*  tx~l  (1  —  t)v~ldt  [9], [10]. 
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where  Q(x)  =  /x+0°  e~v%^dy.  Solving  (31)  with  respect  to  N  we  find  that  the  least 

number  of  samples  (greater  than  M(L  +  NP))  that  guarantees  a  minimum  performance 
confidence  level  in  the  neighborhood  of  the  optimal  point  as  described  by  (24)  and  (25)  is 
approximately  equal  to  N*  given  by  (28).  ■ 

For  the  special  case  of  e  =  0.5  we  have  7  =  0.  In  this  case,  (28)  implies  that  the  least 
number  of  samples  to  achieve  Pr[pv  >  l-(]  >0.5  or  Pr[ps  >  1  — (]  >  0.5  is  approximately 
N *  =  [M(L  +  Np)  -  2]/0  If,  in  addition,  C  =  0.5  then  N*  =  2 [M(L  +  Np)  -  2].  In  plain 
words,  this  example  implies  that  to  be  e  =  50%  confident  that  the  output  variance  of  the 
SMI-MVDR  filter  (or  the  output  SINR  of  the  SMI-MSINR  filter)  will  be  within  £  =  50% 
(-3dB)  of  the  ideal  MVDR  (or  MSINR)  filter  performance  we  need  to  evaluate  the  SMI 
estimators  using  at  least  N*  =  2[M(L  +  NP)  —  2]  data  samples.  We  note  that  1  —  Pr[pv  >  x] 
and  1  —  Pr[ps  >  x]  are  merely  the  cumulative  distribution  functions  (cdfs)  of  the  random 
variables  pv  and  ps,  respectively.  Use  of  the  cdfs  allows  the  evaluation  of  any  statistic  of  the 
corresponding  variables  (e.g.  the  mean  and  beyond).  As  such,  the  work  in  [3]  appears  as  a 
special  case  of  our  developments  in  Theorem  1  (we  recall  that  [3]  evaluates  the  data  record 
size  that  is  necessary  for  the  spatial  SMI-MVDR  beamformer  to  achieve  on  average  (mean) 
a  given  percentage  of  the  optimum  performance) '  In  Table  I  we  evaluate  the  data  record  size 
N*  needed  by  the  SMI-MVDR  filter  and  the  SMI-MSINR  filter  to  operate  with  confidence  e 
within  £  of  the  optimum  performance  for  different  values  of  e  and  (.  A  DS/CDMA  system 
with  a  processing  gain  equal  to  L  =  31  is  assumed.  The  number  of  paths  is  set  Np  —  3  and 
the  receiver  antenna  array  utilizes  M  =  5  antenna  elements.  We  see,  for  example,  that  for 
e  =  0.75  and  (  =  0.25  we  have  N*  =  683.  In  other  words,  SMI  estimation  of  the  MVDR 
filter  based  on  683  samples  guarantees  that  75%  of  the  time  the  performance  of  the  filter 
estimate  will  be  at  most  25%  (-6dB)  worse  than  the  performance  of  the  ideal  filter.  In  Fig. 
1(a)  we  plot  comprehensively  the  probability  Pr[p  >  1  —  £]  as  a  function  of  (  and  the  data 
record  size  N,  while  in  Fig.  1(b)  we  show  the  value  of  N*  as  a  function  of  e  and  (. 

We  emphasize  that  the  domain  of  the  random  variable  p\>  is  different  from  the  domain 
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of  ps  since  py  and  ps  are  based  on  different  input  statistics;  the  former  is  the  ratio  of  the 
output  variance  of  the  ideal  MVDR  filter  over  the  output  variance  of  the  (estimated)  SMI- 
MVDR  filter,  while  the  latter  is  the  ratio  of  the  output  SINR  of  the  (estimated)  SMI-SINR 
filter  over  the  output  SINR  of  the  ideal  MSINR  filter.  The  fact,  however,  that  they  are 
identically  distributed  allows  us  to  treat  them  uniformly  with  the  understanding  that  the 
probabilities  in  the  left  hand  side  of  (26)  and  (27)  (which  correspond  to  different  events)  are 
approximately  equal  to  the  same  value  that  is  given  by  the  right  hand  side  of  (26)  and  (27). 
Similarly,  (28)  identifies  the  approximate  number  of  data  required  to  have  the  same  level  of 
confidence  e  that  the  distance  of  the  different  performance  measures  py  and  ps  will  fall  in  a 
neighborhood  of  radius  C  of  the  optimal  performance  point  py)0pt  =  ps<<ypt  =  1. 

The  data  record  size  N*  in  (28)  guarantees  that  Pr[py  >1-Q>e  and  Pr[ps  >  1— C]  >  e 
hold  true  for  any  given  £  and  e.  N*  was  obtained  by  solving  the  approximate  expressions 
given  by  (26)  and  (27)  with  respect  to  the  data  record  size.  The  approximate  expressions 
(26)  and  (27)  were  obtained  by  using  the  Gaussian  approximation  of  the  binomial  cumulative 
distribution  under  the  assumption  that  R  and  R/+n  are  complex  Wishart  distributed.  To 
examine  both  the  effect  of  this  assumption  on  the  distribution  of  R  and  R/+n  as  well  as  the 
accuracy  of  the  Gaussian  approximation  used,  we  compare  the  approximate  with  the  exact 
(obtained  numerically)  probabilities  Pr[pv  >  1-fl  >  e  and  Pr[ps  >  1-fl  >  e,  respectively. 
In  Fig.  2  we  plot  the  aforementioned  quantities  as  a  function  of  the  data-record-size  N  for 
C  =  0.2.  As  in  Fig.  1,  an  antenna  array  consisting  of  M  =  5  elements  is  assumed,  while  the 
system  processing  gain  L  is  equal  to  31  and  Np  =  3.  The  accuracy  of  the  complex  Wishart 
assumption  and  the  Gaussian  approximation  is  evident.  It  may  be  interesting  to  note  that 
the  probability  density  function  (pdf)  of  py  and  ps  depends  only  on  N,  M,  L  and  Np  and  it 
does  not  depend  on  the  covariance  matrix  R  or  R/+n.  This  observation  implies  that  (28)  is 
valid  regardless  of  the  actual  environment,  i.e.  the  number  of  users,  the  signal  powers,  the 
path  coefficients,  the  path  delays,  the  directions  of  arrival,  etc. 

In  digital  communications  systems  the  ultimate  performance  measure  is  the  bit-error-rate 
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(BER),  that  is  the  probability  of  producing  an  incorrect  decision  on  the  information  bit  bo, 
Pr[b0  ^  &o]-  In  the  following,  we  examine  how  the  performance  measures  in  (24)  and  (25), 
which  are  based  on  the  filter  output  variance  and  the  filter  output  SINR,  respectively,  trans¬ 
late  into  BER  terms.  Under  the  assumption  that  the  received  vector  is  Gaussian  distributed 
(the  same  assumption  that  led  to  the  Wishart  distribution  of  R  and  R/+n),  the  BER  Pe(w) 
exhibited  by  an  arbitrary  linear  filter  w  can  be  expressed  as  follows7: 


where  S(-)  is  given  by 


Pe(w)  ~  Q  (V<S(w)) 

v  a  £o|wHv0|2 
S(w)  — 


(32) 


w*R,+nw-  (33) 

Theorem  2  and  Corollary  1  presented  below  identify  the  relationship  between  the  BER 


performance  and  the  performance  measures  in  (24)  and  (25).  The  proofs  are  given  in  the 
Appendix. 


Theorem  2  Let  the  ideal  MVDR  filter  have  a  BER  region  of  operation  given  by  (10  A,  10  ") 
for  some  A  >  u.  If  for  a  given  (  €  (0, 1)  and  e  €  (0, 1) 

Pr[pv  >  1  -  C]  >  e,  (34) 


then 

P 

where 

10^ 

P 


lOg  P( 5  ( w, 


S  M I  —  MV  DR 


)-P 


10gPe(wMvD^) 


>  a  >  e 


a  = 


(1  -  C)Pc  W  [Q-1  (10^)]3  exp  {-[Q-i(10^]  2/2}  ’ 
logFc(10M)  -ap, 

1 


-(A  +  v),  and 


F((x)  =  Q 


(i  -  OIQ-'W]2 
l  +  ClQ-'W]2 


(35) 

(36) 

(37) 

(38) 

(39) 


Corollary  1  Let  the  ideal  MSINR  filter  have  a  BER  region  of  operation  given  by  (10-A,  10-l/) 
for  some  A  >  v.  If  for  a  given  £  €  (0, 1)  and  e  €  (0, 1) 

_ Pr[ps  >l-(}>e,  (40) 

7We  recall  that  the  data  bits  are  BPSK  modulated. 
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then 


where 


P 


l°g-fe(WMS/7VH) 


>  a 


>  € 


a 


M 

w*) 


A 


lO^y/l  -  Cexpl^Q-^lQ^)]2} 

K<(  10") 
logF,c(10^)  -  a/u, 

1/. 

-(A  +  ^),  and 

q  (yr^c  <r‘w)  . 


(41) 

(42) 

(43) 

(44) 

(45) 


Expression  (35)  shows  that  if  the  data-record-size  is  chosen  to  be  greater  than  N*  in  (28), 
then  the  exponent,  logPe(wSM/_MVDfl))  of  the  BER  performance  of  the  estimated  MVDR 
receiver  will  be  with  probability  at  least  e  in  the  region  (logPe(wWDR),  alogPe(wMVDR)  + 
p)  where  the  scalars  a  and  ft  are  given  by  (36),  (37).  Similarly,  expression  (41)  shows 
that  if  the  data-record-size  is  chosen  to  be  greater  than  N*  in  (28),  then  the  exponent, 
logPe(wSMf_MS;WJj),  of  the  BER  performance  of  the  estimated  MSINR  receiver  will  be  with 
probability  at  least  e  in  the  region  (logPe(wMS,Wfl),  a  log  Pe(wMS, „*)+£),  where  the  scalars 
a  and  0  are  given  by  (42),  (43).  We  note  that  expressions  (35)  and  (41)  are  valid  regardless 
of  the  actual  environment,  i.e.  the  number  of  users,  the  signal  powers,  the  path  coefficients, 
the  path  delays,  the  directions  of  arrival,  etc. 

In  Table  II  we  show  the  region  (log  Pe( w„„) ,  a  log  Pe( w„VDR)+p)  and  (log  P.(w„„M), 
alogPe( 

Wmsinr  )  +  p)  that  the  exponent  of  the  BER  performance  of  the  SMI-MVDR  and 
the  SMI-MSINR  receiver,  respectively,  lies  with  confidence  at  least  e,  for  different  values  of 
C  and  e.  As  in  prior  studies,  L  =  31,  M  =  5,  Np  =  3.  We  see,  for  example,  that  in  the  case 
of  the  SMI-MVDR  filter,  683  samples  guarantee  that  75%  of  the  time  the  output  variance 
will  be  at  most  25%  {-MB)  worse  than  the  output  variance  of  the  ideal  filter.  Thus,  683 
samples  guarantee  that  75%  of  the  time  the  exponent  logPe(wSM/_MVDH)  will  be  in  the  in¬ 
terval  (log  Pe(wMVDR),  0.16  log(Pe(wMVDfi))  -  0.7).  Similarly,  in  the  case  of  the  SMI-MSINR 
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receiver  683  samples  guarantee  that  75%  of  the  time  the  exponent  logPe(wSM/_Ms/WB)  will 
be  in  the  interval  (logPe{wMSJNR),0.7Slog(Pe(wMSINR))  -  O.l). 

We  note  that  while  the  left  end-point,  logPe(wWDfi)  or  logPe(wMSINR),  of  the  SMI- 
MVDR  or  the  SMI-MSINR  performance  range  attains  identical  value,  the  right-end  value  of 
the  performance  range  of  the  SMI-MVDR  receiver  is  larger  than  the  corresponding  value  of 
the  SMI-MSINR  receiver.  This  observation  suggests  the  superiority  of  the  SMI-MSINR  filter 
relative  to  the  SMI-MVDR.  Field  engineers  have  also  observed  this  behavior  in  the  past  and 
occassionally  they  identify  it  as  the  superiority  of  the  SMI- “signal  absent”  estimator  relative 
to  the  performance  of  the  SMI- “signal  present”  estimator.  The  superiority  of  the  SMI- 
MSINR  filter  is  quantified  in  [13]  where  the  pdfs  of  the  output  SINR  and  the  induced  BER 
of  the  SMI-MVDR  and  SMI-MSINR  filter  estimators  are  evaluated  and  a  simple  “estimate- 
and-subtract”  algorithm  is  proposed  for  desired  signal  cancellation  prior  to  filter  estimation. 

IV.  Conclusions 

We  investigated  the  data-record-size  requirements  of  SMI- type  adaptive  algorithms  to  meet 
a  given  performance  objective  in  joint  S-T  interference  suppressing  DS/CDMA  signal  detec¬ 
tion.  We  considered  the  SMI-MVDR  and  the  SMI-MSINR  filter  estimator  and  we  adopted 
the  following  figures  of  merit.  For  the  SMI-MVDR  estimator  the  figure  of  merit  is  the 
ratio  between  the  output  variance  of  the  ideal  filter  (MVDR)  and  the  estimated  filter  (SMI- 
MVDR);  for  the  SMI-MSINR  estimator  the  figure  of  merit  is  the  ratio  of  the  output  SINR 
of  the  estimated  filter  (SMI-MSINR)  over  the  output  SINR  of  the  ideal  filter  (MSINR).  For 
both  cases,  closed  form  expressions  were  derived  that  provide  either  the  data  record  size 
that  is  necessary  to  achieve  a  given  performance  confidence  level  in  a  neighborhood  of  the 
optimal  performance  point  or  the  performance  level  that  can  be  reached  for  a  given  data 
record  size.  This  was  achieved  by  utilizing  close  approximations  of  the  involved  probability 
density  functions.  We  emphasize  that  the  derived  expressions  do  not  depend  on  the  actual 
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value  of  the  ideal  performance  and,  thus,  knowledge  of  this  value  is  not  required.  In  addition, 
the  functional  relationship  between  the  BER  and  the  SINR  exhibited  by  a  linear  receiver 
under  the  Gaussian  input  assumption  enabled  us  to  translate  the  data  record  size  require¬ 
ments  of  the  signal  detection  algorithms  into  BER  terms.  The  practical  significance  of  the 
derived  expressions  lies  in  the  fact  that  the  expressions  are  functions  only  of  the  number 
of  antenna  elements,  the  system  spreading  gain,  and  the  number  of  multipaths,  while  they 
are  independent  of  the  ideal  input  covariance  matrix  which  is  not  known  in  most  realistic 
applications. 


Appendix 


A  Proof  of  Theorem  2 


It  is  straightforward  to  verify  that  the  output  variance  V(w)  and  the  output  SINR  <S(w)  of 
an  arbitrary  linear  filter  w  for  user  0  (distortionless  in  the  vector  direction  v0)  are  related 
through  the  expression 

Thus,  V(w)  =  B«  I1  +  )  ■  («) 


V(wSM/_MVCH)  -  E0(  1  +  - - -  J  and  V(wMVDfl)  =  E0  (\  +  — - - r')  . 

V  ^\^SMI-MVDR)  /  \  S\Wmvdr)J 


Therefore,  the  inequality  py  >  1  -  £  is  equivalent  to 

(1  —  C)S{WMVDr) 


SfasMI-MVDR 

which,  in  turn,  is  equivalent  to 


)> 


(47) 

(48) 

1  C^(wsm/-mvdr)  J  ^  ^ 

since  Q{y/x)  is  a  decreasing  function  of  x.  Solving  Pe( wMVDR)  =  Q  ^S{wMVDR)j  with 
respect  to  S{wMVDR)  for  values  of  Pe(wMVDR)  less  than  0.5,  we  obtain 

S{vmvdr)  =  [Q-1(Pe(wMVi,fl))]2  (50) 


Pe(  W 


SMI-MVDR 


)<Q 


1  "t"  C^(WMVDp) 


I  (1  C)${WsMI-MVDr) 
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where  Q-1(‘)  is  the  inverse  function  of  Q(-).  Using  (50)  in  (49)  we  see  that  (34)  is  equivalent 


Pr  P  (xv  \  O  (  / ~  CKQ  1(-Pe(wMVDfi)))2 

Pt  |F.(w <  «  I  y  i  + 

(51)  can  be  written  in  a  more  compact  form  as  follows: 


Pf  [Pei^SMI-MVDF.)  •F^(-f>e(WMVDfi))]  ^  C 


where 


Fc(x)  =  Q 


f(i-C)(Q-W 
i  +  C(Q-1(*))2 


The  function  Fc(rr)  in  (53)  is  non-linear.  However,  logFc(a;)  can  be  approximated  closely 
by  a  linear  function  of  logx,  i.e.  logF^(x)  ~  a  log  a;  +  fi.  For  a  BER  region  of  operation  of 
the  ideal  MVDR  receiver  of  the  form  (10~A,  10_1/)  for  some  A  >  u,  the  parameters  a  and  ft 
in  the  above  approximation  can  be  chosen  as  the  coefficients  of  the  first  order  Taylor  series 
expansion  of  the  function  logF^lO1)  about  p  =  |(A  +  u): 

in^  T  /q-cMQ-Hio^V  r::n  f  1 

10  VV  l+ctQ-Hio^)]2  J  e  pl  zq+qQ-Mw*)]2)/  ,  * 

Q  "  (1  —  C)^c(10M)[Q~1(10*1)]3exp{— [<2_1(10^)]2/2}’  1 

£  =  logFc(10^)  -ap.  (55) 

Thus,  F{(x)  can  be  approximated  by 

F{(x)  =  10  V.  (56) 

In  Fig.  3  we  examine  the  accuracy  of  the  approximation  of  F^(x)  by  (56).  In  Fig.  3(a) 

we  plot  F^(x)  along  with  its  approximation  F^ (x) ,  while  in  Fig.  3(b)  we  plot  the  relative 
error  of  the  approximation  | F^ (x)  —  F^(x)\/ Fq(x)  for  two  different  values  of  C>  C  =  0-2  and 
(  =  0.4.  The  operating  region  of  the  receiver  is  assumed  to  be  [10-3, 10-1]  which  implies 
that  p  =  —  2.  The  accuracy  of  the  approximation  F^(x)  is  evident.  Finally,  replacing  F^(x) 


by  F^(x),  (52)  becomes 


pr  i°g-Pe(wSM7-MyDR)  -  H  >  a  >  € 

logFe(wMVDfi) 
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B  Proof  of  Corollary  1 


Following  the  same  approach  as  in  the  proof  of  Theorem  2,  we  see  that  Pr[ps  >  1  —  £]  >  e 
is  equivalent  to 


Pr  [n(»s„,-„s„,)  <  Q  (v^C  Q-'  (P,(w„s, „„)))]  >  e.  (58) 

If  we  define 

f.cW  =  Q  (v^C  «-*(*)) ,  (59) 

we  can  express  (58)  in  a  more  compact  form  as 

Pt  [-Pe(wSAf i-msinr)  <  ( P p(wM5/iVJl))]  >  €.  (60) 


The  logarithm  of  the  non-linear  function  F*^(x)  can  be  approximated  closely  by  a  linear 
function  of  logx,  i.e.  log  F^(x)  —  o  log x  +  fi.  For  a  BER  region  of  operation  of  the  ideal 


MSINR  receiver  of  the  form  (10  A,  10  v)  for  some  A  >  v,  the  parameters  a  and  /?  in  the  above 
approximation  can  be  chosen  as  the  coefficients  of  the  first  order  Taylor  series  expansion  of 


the  function  logF.f  (10*)  about  p  =  J(A  +  v): 

IQVl  -  CexpdfQ-^lQ^)]2} 


P  =  logFc(10'1)  -  ap. 
Thus,  F*((x)  can  be  approximated  by 


(61) 

(62) 


F.c(x)  =  10V. 


(63) 


In  Fig.  4  we  examine  the  accuracy  of  the  approximation  of  F*c(x)  by  F,c(x).  In  Fig.  4(a) 
we  plot  F,c(x)  along  with  its  approximation  F,f(x),  while  in  Fig.  4(b)  we  plot  the  relative 
error  of  the  approximation  |F»f(x)  -  F,c(x)|/F,c(x)  for  two  different  values  of  C,  C  =  0.2  and 
C  =  0.4.  The  operating  region  of  the  receiver  is  assumed  to  be  [10-3, 10-1]  which  implies 
that  p  =  -2.  The  accuracy  of  the  approximation  F*c(x)  is  evident.  Finally,  replacing  F,c(x) 
by  F*^(x),  (60)  becomes 


log  F ;(w; 


SMI— MSI  NR 


)~P 


logPe(w 

MSINR ) 


>  a 


>  e. 


(64) 
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Table  I 

Data-record-size  requirements  of  SMI-MVDR  and  SMI-MSINR 

I - SIGNAL  DETECTION  ALGORITHMS _ 

Space-time  product  { e  I  Data  record  size 

M  x  {L  +  Np) _  n* 


5  x  (31  +  3) 

0.25  (-6 dB) 

0.75 

683 

■BilXHi 

0.25  (— 6dB) 

0.9 

712 

5  x  (31  +  3) 

0.1  (— lOdB) 

0.9 

1793 
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requirement  N*  as  given  by  Theorem  1,  Part 


Relative  Error  F.r(x) 


Figure  3:  (a)  Approximation  of  Fc(z)  by  Fc(x).  (b)  Relative  error  of  the  approximation. 
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ABSTRACT 

A  doubly  optimal  binary  signature  set  is  a  set  of  binary  spreading  sequences  that  can  be  used  for  code  division 
multiplexing  purposes  and  exhibits  minimum  total-squared-correlation  (TSC)  and  minimum  maximum-squared- 
correlation  (MSC)  at  the  same  time.  In  this  article,  we  focus  on  such  sets  with  signatures  of  odd  length  and  we 
derive  closed-form  expressions  for  the  signature  cross-correlation  matrix,  its  eigenvalues,  and  its  inverse.  Then, 
we  derive  analytic  expressions  for  (i)  the  bit-error-rate  (BER)  upon  decorrelating  processing,  (ii)  the  maximum 
achievable  signal-to-interference-plus-noise  (SINR)  ratio  upon  minimum-mean-square-error  (MMSE)  filtering, 
and  (Hi )  the  total  asymptotic  efficiency  of  the  system.  We  find  that  doubly  optimal  sets  with  signature  length  of 
the  form  4m +1,  m  —  1, 2, . . . ,  are  in  all  respects  superior  to  doubly  optimal  sets  with  signature  length  of  the  form 
4m-l  (the  latter  class  includes  the  familiar  Gold  sets  as  a  small  proper  subset).  “4m+l”  sets  perform  practically 
at  the  single-user-bound  (SUB)  after  decorrelating  or  MMSE  processing  (not  true  for  “4m -1”  sets).  The  total 
asymptotic  efficiency  of  “4m+l”  sets  is  lower  bounded  by  \  for  any  system  user  load.  The  corresponding  lower 
bound  for  “4m-l”  sets  is  zero. 

Keywords:  Binary  sequences,  code  division  multiaccess,  codes,  Gold  codes,  signal  design,  spread  spectrum 
communication,  Welch  bound. 

1.  INTRODUCTION  AND  BACKGROUND 

In  code  division  multiplexing  systems,  such  as  the  present  wireless  direct-sequence  code-division-multiple-access 
(DS-CDMA)  technology,  distinct  data  streams  are  modulated  by  individual  signature  patterns  and  sent  over  a 
common  in  time  and  frequency  channel.  The  success  or,  more  general,  performance  of  the  multiplexing  scheme 
(or  multiple-access  scheme  depending  on  the  point  of  view)  depends  on  the  specific  assigned  set  of  signatures. 
Two  critical  signature  set  metrics  are  the  total-squared-correlation  (TSC)  and  the  maximum-squared-correlation 
(MSC)  among  signatures.1-11  Intuitively,  both  metric  values  need  to  be  kept  low. 

In  Ref.  8,  we  derived  new  lower  bounds  on  the  TSC  of  binary  antipodal  signature  sets  for  all  possible 
combinations  of  number  of  data  streams/users  K  and  signature  length/processing  gain  L  and  we  proved  the 
tightness  of  the  new  bounds  for  all  K,  L  except  (i)  L  —  K  =  \  (mod  4),  (ii)  L  =  K  =  2  (mod  4),  (Hi) 
L  +  1  =  K  =  2  (mod  4),  and  (iv)  K  +  1  =  L  =  2  (mod  4).  Ding,  Golin,  and  Klqve9  established  the  tightness  of 
these  bounds  also  under  Cases  (ii)-(iv))  All  proofs  of  tightness  in  Ref.  8,  as  well  as  in  Ref.  9,  are  by  construction 
and  present  us  with  simple  algorithms  for  the  design  of  minimum-TSC  optimum  binary  signature  sets  based 
on  Hadamard  matrix  transformations.  The  familiar  Rademacher-Walsh-Hadamard  code  sets12,13  with  L  =  2m, 
m  =  1, 2, . . . ,  and  K  <  L  and  their  extension  to  L-  4m,  m  =  1,2,...,  and  K  <  L,  the  m-sequence  sets14  with 
L  =  2m  —  1,  m  =  3, 4, . . . ,  and  K  <  L,  and  the  Gold  sets15  with  L  =  2m  —  1,  m  =  5, 6, . . . ,  and  K  <  L  +  2, 
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of  Scientific  Research  under  Grant  F49620-01-1-0176. 
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fThe  case  K  =  L  =  1  (mod  4)  remains  open.  Ding,  Golin,  and  Klpve9  showed  that  our  bound  in  Ref.  8  is  tight  for 
K  =  L  =  5  or  13,  but  not  for  K  =  L  =  9.  What  happens  when  K  =  L  >  17  is  an  open  question. 


can  all  be  seen  as  specific  output  examples  of  the  design  nrocednre  in  a  o  t  o  f  , , 
that  for  “underloaded  systems”  (K  <  L)  the  minimum-TSC  optimal  designs’ in  Rrff  9  TnT  T  * ' 

maximum-squared-correlation  (MSC)  at  the  same  time  w»  -«n  t-i.  t  S  i  K  *'  8’ 9  h  ’  n  fact’  minimum 
both  in  the  minimum-TSC  ^  ^  d°Ubly  °ptimal:  °Ptiraal 

of  the  results  in  Ref.  8, 9,  we  recognil tllrldS uJtZ  theoretically  their  multiaccess  behavior.  In  view 
form  4m+l,  ,-1,1 . and  sets  with  signature  length!  the  Sfm  4m-Hi  =  ?  TT"'  ‘“^Vr  M 

(bit-«ror-rate  o^BER,  ^flhe^econelating1  detectoT  llLthe"3^'0  ex*)re®s*°1^  kt  Vt/the  iwobabiltty  of  error 
interference-plus-noise  ratio  (SINR)  of  the  minimum-mfan.  6  /MSE)  311(1  output  signal-to- 

asymptotic  efficiency  of  the  system  Direct  analvtir  square-error  (MMSE)  linear  filter,  and  (Hi)  the  total 
Decorrelation  of  “4m+I”  °f  '*?*' »*  — 

“4m— 1’’  codes.  “4m+l”  codes  attain  hieher  qTNR  f*  *  d  \b^B)  BER  performance.  This  is  not  so  for 
(therefore,  smaller  occupied  bandwidth)  than  “4m-r  lodes“  “4m+l”^tIfE  4'  tor,s.m.aller  processir,8  gain 

efficiency  than  “4m-1”  codes  and  guarL.ee  a  loZ,  bound  of  *  t+l  u^'H  *?■ I  T  7“pt°‘‘C 
corresponding  bound  for  “4m-l”  codes  is  zero.  e  7  1  d  contrast,  the 

The  presentation  of  this  material  is  organized  as  follows.  In  Section  9  aft»r  *  j 
and  definitions,  we  derive  a  few  essential  DroDerties  of  the  rlonhi  1  '•  ntroduce  pertinent  notation 

performance  of  code  division  mul  w“r£ '  Si  S  uS  such  so  T*™*  **  ^  Th' 

Section  3.  In  Section  4,  we  present  a  few  nnSal  el mnS  ,Td  P  w‘"g  “d“  “  e*amined  theoretically  in 
the  theoretical  developments  of  Section  3.  Some  concluding  remarble  gilnlfetll”'™1'  ‘"t<'rPr" 

2.  NOTATION,  DEFINITIONS,  AND  A  FEW  DERIVATIONS 

We  consider  a  binary  antipodal  signature  set  with  K  normalized  signatures  s*  €  /i-U*  k  -  i  2  K 
where  L  is  the  DS-CDMA  system  processing  gain;  we  represent  this  set  by  J  lVK  ’signal  mairt 
[sit S2»  •  •  • , s/c]  •  The  signature  crvss-correlation  matrirTl  ic  Hofin  a  . 

correlations  (inner  products)  between  the  signatures  in  the  set:  “  6  matrUC  that  contains  311  cr°ss- 

R  =  STS. 

Below,  we  derive  some  useful  expressions  for  R,  its  eigenvalues  X  X  \  ,  .  . 

an  underloaded  (K  <  L)  doubly  optimal  net  of  odd  processing  g!  n'  w’  exile  fhaT'*  *'  ’’T  S  “ 
and  L  =  4m  +  1,  m  =  1, 2, . . . ,  separately  B  8  examine  the  two  cases  L  =  4m  -  1 

1)  L  =  4m  -  1 

“l fir.r?'  that  f°r  ‘  d°“bly  °Ptimal  Sig“Ure  S  »“b  P— Of  thefbrm 

a-£fiUr--r  (2) 

where  the  vector  s  €  {±^}  depends  on>  the  specific  signature  matrix  S  and  In  denotes  the  K  x  K  nw. 
matrix.  Then,  the  If  orders  e^envalues  of  R  given  the  form  of  a  Jr  x  I  vec  Ja  ^  ££  *  ^ 


A  =  — +  1  &  +  1  L  -  K  +  llT 

L  L  L  ’  L -  ' 


The  Inverse  of  R  can  be  obtained  from  (2)  using  the  Matrix  Inversion  Lemma-  (Woodbury's  identity): 


R  1  =  — —  IK  4 - — _ J^s  (i  _  _T _ 4 

L±  1  ^  Z,  +  l1/cS  l 1  s  14 


tIks)  "’rfr1" - ltt'* + 


*For  example,  if  S  is  a  Gold  signature  set  (L  =  2m  -  1,  m  =  5,6, . .  .),18  then  s  =  [1, 1,  i]^ 


(i+l)(I-A'+l)SsT-  (4^ 


LVdSr°PttoaI  signature  matrix  S  with  processing  gain  of  the  form  L  -  4m  +  1,  m  =  1,2 . «e  can 

show  using  the  findings  in  Ref.  8, 9  that  ... 

R  =  — —Ik  +  ssT  W 

where  the  vector  s  €  {±^}*  depends  on  the  specific  signature  matrix  S.  The  K  x  1  vector  A  with  the  ordered 
eigenvalues  of  R  is  {L  +  K-l  L-l  if  (6) 

A=  L  ’  L  L  j 


The  inverse  of  R  is  obtained  from  (5)  using  the  Matrix  Inversion  Lemma: 


R-*  = 


ijlrrs  (l  +  J~j1kS)  1 


(L-1)(L  +  K-1)' 


3  ODD-LENGTH  DOUBLY  OPTIMAL  CODES:  THEORETICAL  PERFORMANCE 
a.  uuu  lieji  ANALYSIS 

SSsSisrrif; 

r  =  y/EkSk  4-  n  ^ 

k= 1 

where  b k  €  {±1}  is  the  fcth  user  information  bit,  Ek  denotes  received  energy  per  bit  and  n  represents  the  dltered 
ST vector 1  If \  is  the  power  spectral  density  of  the  underlying  AWGN  process,  then  n~Af  (0txi,  and 

7fc  ^  ^  is  the  fcth  ^  signal-to-noise  ratio  (SNR)  per  bit.  We  define  the  K  x  K  diagonal  matrix  that  consrsts 

of  the  A  user  energies  E  =  diag  (ft,  ft,  •  •  • ,  ftc)  and,  similarly,  the  diagonal  matrix  T  =  diag  (7i,72,  •  •  •  ,7 k) 
that  contains  the  K  user  SNRs.  In  the  sequel,  we  evaluate  theoretically  the  performance  of  this  code  ivision 
multiplexing  system  when  doubly  optimal  spreading  codes  of  odd  processing  gam  are  used. 

3.1.  Decorrelating  Receivers 

The  decorrelating  linear  filter  (or  decorrelator)^  “  output  1,  the  ML  linear  estimate  of  the  information  bit  b„. 

The  fcth  user  decorrelator  is  given  by  ,  n  (9) 

wfc  —  S  [R  JfcthcoIumn 

where  R  is  the  signature  crose-correlation  matrix*.  It  ia  known  that  the  BER  of  the  decorrelato,  followed  by  a 
sign  detector  is  ,  - \ 


where  O(x)  =  f°°  -4=e~t2/2dt.  Therefore,  the  BER  performance  of  the  fcth  decorrelator  is  determined  by  the 

fcth  diagonal  element  of  the  inverse  signature  cross-correlation  matrix  R"1  (and  the  fcth  user  SNR  of  course). 

the  use  of  odd-length  doubly  optima,  spreading  codes*.*”  and  ?  :  then  **  on 

the  decorrelator  BER  performance.  We  need  to  study  the  two  cases  L  =  4m  -  1  and  L  —  4m  + 1,  m  -  1,2, ... , 
separately. 

iln  our  notation,  if  A  is  an  m  x  n  matrix,  then  [A]uh  colura„  ,  fc  =  1,2, .  • is  the  m  x  1  vector  defined  by  the  fcth 
column  of  A.  [A]j  (c ,  j  =  1, 2, . . .  ,m,  fc  =  1, 2, . . . , n,  is  the  (j,  fc)th  element  of  A. 


1)  L  =  4m  -  1 
From  (4)  we  obtain: 


[R  kfc  L  +  l  +  (I  +  1)(L-  K+l) 

2)  L  =  4m  -+  1 
From  (7)  we  obtain: 


1  + 


tf-l 

(I  +  1)(L-K  +  1)- 


[R 


-1 


1  =  — 
Jfc.fc  L—l 


L2 


(L  -  1)(L  +  K  -  1) 


(s[fc])2 


1  + 


a:-i 


(ii) 


(12) 


We  note  that  the  [R-1]fc  k  =  1  value  in  (10)  corresponds  to  single-user-bound  (SUB)  performance  and  we 
recall  that  for  any  K,  L  and  signature  set  S  for  which  the  decorrelator  exists,  [R_1]fc  k  >  1.  From  (11),  it  is 
interesting  to  observe  that  when  we  use  doubly  optimal  signatures  with  L  =  Am -l,  then  the  coefficient  [R_1]fc  k 
becomes  much  greater  than  1  when  the  system  is  heavily  loaded  ([R-1]fc  fc  =  1  +  _ ►  2).  On  the 

other  hand,  the  coefficient  [R  *]  k  k  in  (12)  for  L  =  4m  +  1  doubly  optimal  signatures  is  always  kept  close  to 
1,  even  for  high  user  loads  ([R_1]fc  ^1  +  ^  1).  As  a  result,  we  conclude  that  (4m+l)-long 

doubly  optimal  codes  maintain  near  SUB  performance  even  for  heavily  loaded  systems,  while  (4m-l)-long 
doubly  optimal  codes  fail  (to  place  this  conclusion  in  the  broad  context  of  pertinent  past  literature,  we  recall 
that  the  Gold  codes15  are  in  fact  a  small  proper  subset  of  the  (4m-l)-long  doubly  optimal  codes8-9’11).  Later, 
in  Section  4,  we  present  for  illustration  purposes  some  numerical  BER  performance  plots  for  the  decorrelator 
when  (4m-l)  or  (4m+l)-long  doubly  optimal  codes  are  utilized. 


3.2.  MMSE  Receivers 

The  MMSE  (or  Wiener)  filter20-21  is  the  linear  filter  that  minimizes  the  MSE  between  its  output  and  the  true 
information  bit.  It  can  be  shown  that  the  MMSE  filter  maximizes  the  SINR  at  its  output.  The  fcth  user  MMSE 
filter  is  given  by 

W(MMSE)  =  r-1^ 

where  R^  is  the  auto-correlation  matrix  of  the  received  input  vector  r  : 


R A  =  E  {rrT}  =  SESt  -I-  ^1^. 
It  is  well  known  that  the  MSE  of  the  fcth  user  optimum  MMSE  filter  is 

MMSE*  =  1  -  EkslR-'sk 
and  the  fcth  user  maximum  output  SINR  value  is 

1 


SINR*  = 

*  MMSE*, 

We  now  apply  the  Matrix  Inversion  Lemma  to  (14)  to  obtain: 


-1. 


R;'  -  -  ^SEi  +  ^E,srsE!)"  EisTwa 


(13) 

(14) 


(15) 


(16) 


(17) 


(18) 


Therefore, 


Next,  we  use  the  Matrix  Inversion  Lemma  to  calculate  the  inverse  of  E  4-  ^R"1  : 

(e  +  a H-) ' "  =  -  ini  (l*  +  E*  £l«»)  El  Ar. 

Prom  (18),  (19),  we  conclude  that 

StR^S=  ^E  +  ^R-1^  \ 

We  note  that  s^R  ^Sk  =  [STRi41S]fc  fe .  Then,  from  (15)  and  (20)  we  obtain: 

(E  +  ^R-i)  =1_  ^  +  ^E-iR-iE-iy1j 

We  recall  the  definition  T  =  diag  (71,  72,  ■  ■  ■ ,  Ik)  =  diag  (^,  . . . ,  =  ^E.  Then, 

MMSEfc  =  i-  [(iA:  +  r-iR-1r-^~1J  . 


MMSE*  =  1  -Ek 


k,k 


(19) 


(20) 


(21) 


(22) 


In  the  following,  we  consider  the  possible  use  of  odd-length  doubly  optimal  spreading  codes8, 9' 11  in  conjunction 
with  MMSE  receivers.  As  in  the  previous  decorrelator  subsection,  we  examine  the  two  cases  L  =  4m  -  1  and 
L  =  4 m  +  1,  separately. 

1)  L  =  4m  -  1 

From  (22)  and  (4)  we  are  able  to  obtain  the  following  closed-form  analytic  expression  for  the  fcth  user  MMSE  as 
a  function  of  the  system  parameters  L,  K,  and  71,  72,  . . . ,  7 k  when  any  doubly  optimal  signature  set  of  length 
L  =  4m  -  1,  m  =  1, 2, . . . ,  is  used  (we  mention  again  that  the  familiar  Gold  sets  fall  exactly  under  this  class): 


MMSE*  =  1  -  - 

I+T  +7* 


1 


(_4r+7,)[g±a^±ii+g_i 


(23) 


The  detailed  derivation  (proof)  of  (23)  is  omitted  due  to  lack  of  space.  Since  SINR*  =  * 

conclude  that  _  MMSE* 


STNTR,  -  1 

1  —-7* _ 

f,_  .  ) 

- 1. 

T+T+7fc 

(xtl+*0 

<L  +  1)(L-/C  +  1)  rK  1 

\j 

1  (cf.  (16)),  we 
(24) 


2)  L  =  4m  +  1 

Similarly,  from  (22)  and  (7)  we  obtain  the  following  closed-form  expression  for  doubly  optimal  sets  of  signatures 
with  length  L  =  4m  +  1,  m  =  1,2, - The  derivation  is  omitted. 


MMSEfc  =  1  -  —r^ - 

tt  +  lk 


We  conclude  that  (cf.  (16)), 


1 


\ 


(z^i+7*) 


STNR -  1 

X  — 

i,i _ ,  1 

—  1. 

r^r+7fc 

^  (rtr+7fc) 

LL-l)(L+K-l)  ^ K  ! 

y 

(25) 


(26) 


The  analytic  expressions  derived  in  this  section  can  be  used  for  the  direct  evaluation/comparison  of  the  SINR 
(or  MMSE)  performance  of  odd-length  doubly  optimal  spreading  codes.  As  a  case  study,  we  proved  the  following 
proposition  that  holds  under  “perfect  power-control.”  The  proof  is  omitted  due  to  lack  of  space. 

Proposition  1.  Consider  all  doubly  optimal  spreading  codes8,9, 11  of  length  L\  =  4 m  -  1  and  L2  =  4m  4- 1,  m  = 
1,2, _ Assume  that  7*  =  7,  A:  =  1,2, . . .  and  consider  MMSE  filter  receivers.  Then ,  SINR^2^  >  SINR^1^ 


(or,  equivalently ,  MMSE^2^  <  MMSE^1^  if  and  only  if 


L>2  >  L\  — 


7  +  1 


(K-2). 


□ 


Proposition  1  identifies  the  minimum  processing  gain  L2  of  the  form  J^m+1  that  is  required  to  achieve  better 
SINR  (or  MMSE)  performance  than  a  given  processing  gain  L\  of  the  form  ^m-I.  For  example,  set  K  =  30 
and  7  =  lOdB.  If  L\  =  63,  then  for  any  L2  >  37.5  (i.e.  L2  =  41,  45,  49,  53,  ...)  the  corresponding  doubly 
optimal  codes  outperform  in  SINR  (or  MMSE)  all  63-long  codes  (that,  we  recall,  include  the  63-long  Gold 
codes).  In  general,  Proposition  1  implies  that  (4m+l)-long  spreading  codes  can  achieve  a  higher  output  SINR 
than  (4m- 1)- long  codes  and  simultaneously  occupy  a  significantly  smaller  bandwidth. 


In  Section  4,  we  present  a  few  numerical  studies  that  illustrate  the  maximum  SINR  performance  of  odd-length 
doubly  optimal  spreading  codes  derived  above. 


3.3.  Total  Asymptotic  Efficiency 

The  total  asymptotic  efficiency22,23  of  a  DS-CDMA  system  equals  the  determinant  of  the  signature  cross¬ 
correlation  matrix  |R|.  Since  m  =  n£Li  A*,  where  A*,  k  =  1,2,..., AT,  are  the  eigenvalues  of  R,  we  can 
use  our  eigenvalue  findings  in  (3)  and  (6)  to  obtain  closed-form  expressions  for  the  total  asymptotic  efficiency  of 
CDMA  systems  that  utilize  odd-length  doubly  optimal  spreading  codes: 


L  =  4m  -  1,  m  =  1,2, . . . , 
L  —  4m  -f  1,  m  =  l,2, — 


(27) 


In  the  following  proposition  we  show  that  DS-CDMA  systems  with  (4m+l)-long  doubly  optimal  spreading 
codes  have  strictly  greater  total  asymptotic  efficiency  than  systems  with  (4m-l)-long  codes.  The  proof  is 
omitted  due  to  lack  of  space. 

PROPOSITION  2.  Consider  code  division  multiplexing  systems  with  doubly  optimal  signature  sets  of  length  L\  = 
4m  -  1  and  L2  =  4m  +  1,  m  =  1,2,...,  and  K  >  1  users.  Then ,  the  total  asymptotic  efficiency  of  L2— type 
systems  is  strictly  greater  than  L\—type  systems:  |Rt2|  >  |R h\  ■  □ 

It  may  be  even  more  interesting  to  examine  the  asymptotic  behavior  of  the  total  asymptotic  efficiency  |R|  as 
the  processing  gain  and  the  number  of  users  increase  (L  — ►  00  and  K  — ►  00)  while  their  ratio  (user  load)  ^  is 
kept  constant.  Below*  we  establish  formally  that  (4m+l)-long  codes  are  significantly  superior  to  (4m-l)-long 

codes  in  terms  of  the  asymptotic  behavior  of  |R|  for  the  same  user  load  a  =  We  present  our  theoretical 
developments  in  the  form  of  the  following  proposition  whose  proof  is  omitted. 


Proposition  3. 

(a)  Let  Ri  be  the  signature  cross- correlation  matrix  of  a  doubly  optimal  spreading  code  set  with  processing  gain 
of  the  form  L  =  4m  -  1,  m  =  1, 2, . . . ,  and  user  load  a  €  (0, 1] . 


(i) 

and 


Fix  the  user  load  a.  Then,  |Ri|  is  a  monotonically  decreasing  function  of  L,  L> 


lim  |Ri|  =  (1  -  a)ea. 


1 

a 


.  For  L  =  A,  |R,|  =  1 


(28) 


(ii)  Fix  the  processing  gain  L.  Then,  |Ri|  is  a  monotonically  decreasing  function  of  a,  £  <  a  <  1,  and 


a  =  i 

0=1. 


(29) 


(b)  Let  R2  be  the  signature  cross-correlation  matrix  of  a  doubly  optimal  spreading  code  set  with  processing  gain 
of  the  form  L  =  4m  +  1,  m  =  1, 2, . . . ,  and  user  load  a  €  (0, 1] . 

(i)  Fix  the  user  load  a.  Then,  |R2|  is  a  monotonically  decreasing  function  of  L,  L  >  For  L  =  £,  |R2|  =  1 


and 


lim  |R2|  =  (1  +  oi)e~a. 

L— >oo 


(30) 


(ii)  Fix  the  processing  gain  L.  Then,  |R2|  is  a  monotonically  decreasing  function  ofa,j;<a<  1,  and 


a  =  t 


1 
L 

a  =  1. 


(31) 

□ 


For  example,  according  to  Proposition  3,  if  we  fix  the  user  load  a  to  1  (fully  loaded  system)  and  we  let  the 
processing  gain  L  increase  to  infinity,  the  total  asymptotic  efficiency  |Ri|  of  (4m-l)-long  codes  converges  to 
0  while  the  total  asymptotic  efficiency  |R2|  of  (4m+l)-long  codes  converges  to  In  fact,  since  (1  -a)e  < 
(1  +  a)  e~a  Vq  €  (0  1] ,  Proposition  3  shows  that  (4m+l)-long  codes  achieve  higher  total  asymptotic  efficiency 
than  (4m-l)-long  codes  as  L  oo  for  any  load  a  €  (0, 1] .  This  conclusion  is  showcased  below  in  the  form  of 


a  corollary. 

Corollary  1 .  Consider  two  doubly  optimal  sets  with  processing  gain  of  the  form  L\  =  4m  - 1  and  L2  =  4m  + 1 , 
m  =  1,2, ... ,  and  signature  cross- correlation  matrices  Ri  and  R2,  respectively .  If  the  two  sets  have  the  same 

fixed  user  load  a  €  (0, 1] ,  then 


^lirn^  |Ri|  =  (1  -  a)  eQ  <  (1  +  a)  e  “  =  ^m^  |R2| . 


(32) 


□ 


In  the  context  of  doubly  optimal  sets  with  finite  (non-asymptotic)  length  L  and  load  a  €  (0, 1] ,  Proposition  3 
offers  the  following  conclusion.  While  in  general  the  total  asymptotic  efficiency  |R|  of  a  signature  set  is  between 
0  and  1  (in  fact,  these  numbers  are  tight  bounds  for  |R|  when  the  processing  gain  is  of  the  form  L  =  4m -1  (ct. 
Proposition  3)),  (4m+l)-long  doubly  optimal  spreading  codes  guarantee  a  minimum  total  asymptotic  efficiency 
|R|  =  1  ~  .73  for  any  processing  gain  L  =  4m  +  1  and  any  user  load  a  €  (0, 1] .  We  present  this  finding  below 

as  a  second  corollary  to  Proposition  3. 

Corollary  2.  The  total  asymptotic  efficiency  |R|  of  doubly  optimal  sets  with  processing  gain  of  the  form 
L  =  4m  +  1,  m  =  1, 2, . . . ,  is  lower  bounded  by  f  : 


-  <  |R|  <  1. 

e 


(33) 

□ 


4.  NUMERICAL  EXAMPLES  AND  SIMULATION  STUDIES 

In  this  section  we  present  a  few  numerical  examples  that  may  help  us  visualize  the  theoretical  developments 
of  Section  3.  The  analysis  of  Section  3  applies  to  any  doubly  optimal  spreading  code  set  with  odd  processing 
gain.  Here  we  choose  to  compare  the  performance  of  31-long  and  63-long  doubly  optimal  Gold  codes  to  the 
performance  of  33-long  and  65-long  doubly  optimal  codes.8- 9- 11 

In  Fig.  1  we  consider  underloaded-by-one  doubly  optimal  sets,  that  is  (a)  a  Gold  set  with  Li  =  31  and 
K,  =  30  (b)  a  doubly  optimal  set  with  L2  =  33  and  K2  =  32,  (c)  a  Gold  set  with  L[  =  63  and  K[  =  62,  and 
(d)  a  doubly  optimal  set  with  L'2  =  65  and  K'2  =  64.  We  plot  the  BER  curve  of  the  single-user  decorrelating 
detector  as  a  function  of  the  received  (input)  SNR  lk  of  the  user  of  interest  k  directly  from  (10),  (11),  and  (12). 
The  single-user  bound  (SUB)  is  also  included  for  reference  purposes.  The  loads  35  and  ^  of  the  4m+l  codes 
are  higher  than  the  corresponding  loads  §  and  §§  of  the  “4m-l”  (Gold)  codes.  Still,  the  decorrelating  detector 


'np“ SNR  ,or  31  <Go,d>' 33> 63  (g°u)-  "d  ^  ^ 


opfimTl  sets^nhSO^^et^dfir"  MMSE  fil‘er  ver^s;nput  SNR  for  31  (Gold),  33,  63  (Gold),  and  65-long  doubly 
optimal  sets  with  30,  32,  62,  and  64  users,  respectively.  Perfect  user  power  control  is  assumed.  7 


performs  practically  at  the  SUB  for  the  former  but  not  for  the  latter.  We  conclude  that  we  achieve  almost  SUB 
performance  with  a  linear  decorrelator  if  the  processing  gain  is  of  the  form  4m+l  and  the  signature  set  is  doubly 
optimal.  This  is  not  true,  however,  for  (4m- 1) -long  doubly  optimal  spreading  codes  (that  include  the  Gold 

ctmd  Flg  ,2,  We  f131"1^111  the  same  set’uP  88  in  Fig-  1  and  we  use  expressions  (24)  and  (26)  to  plot  the  outrmt 
SINR  of  the  single-user  MMSE  filter  as  a  function  of  the  input  SNR  which  for  simplicity  is^sumed  to  be  fhe 


same  for  all  users  (power  control).  For  (4m+l)-long  codes  the  SNR  “loss”  due  to  multiuser  interference  is 
practically  non-existent.  This  is  not,  however,  the  case  for  (4m-l)-long  (Gold)  codes.  Notice  that  for  an  inpu 
SNR  of  20dB  the  output  SINR  of  the  MMSE  filter  for  31  and  63-long  doubly  optimal  (Gold)  codes  is  about 
18.6dB,  while  it  is  19.9dB  for  33  and  65-long  “4m+l”  doubly  optimal  codes. 

Numerical  studies  on  the  total  asymptotic  efficiency  |R|  are  presented  in  Figs.  3  and  4  where  we  plot  |R|  as 
a  function  of  the  processing  gain  L  for  a  fixed  load  a  for  both  (4m-l)-long  and  (4m+l)-long  doubly  optima^ 
codes.  In  Fig.  3,  a  is  set  to  0.5  (half-loaded  system).  In  Fig.  4,  a  is  set  to  1  (fully  loaded  system).  In  both 
studies,  the  superiority  of  (4m+l)-long  codes  is  apparent.  It  is  interesting,  for  example,  to  observe  in  Fig. 


Figure  5.  BER  of  single-user  MMSE  filter  versus  total  input  SNR  for  63-long  (Gold)  and  65-long  doubly  optimal 
(asynchronous  link,  three  unit-gain  paths). 
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the  convergence  of  |R|  to  f  ~  .82  and  ^  s  .91  for  “4m -1”  and  “4m+l”  doubly  optimal  codes,  respectively, 

as  proved  in  the  previous  section.  Similarly,  we  observe  in  Fig.  4  the  quick  convergence  of  IRI  to  0  and  -  ~  73 
respectively.  c  ’  ’ 

Finally,  a  simulation  study  that  examines  the  performance  of  a  63-long  Gold  and  a  65-long  doubly  optimal  set 
for  asynchronous  transmissions  over  multipath  channels  is  presented  in  Fig.  5.  We  consider  the  presence  of  3  fixed 
chip-spaced  paths  (of  equal  gain  set  to  1)  for  each  user  signal  and  we  assume  independent  uniformly  distributed 

^8pff  f°r  Cach  interfering  signal  with  resPect  t0  the  user  signal  of  interest.  We  plot  the  average 

BLR  of  the  MMSE  filter  receiver  (of  length  equal  to  the  multipath-extended  processing  gain  L  +  3  -  1  =  L  +  2) 
as  a  function  of  the  total  received  (input)  SNR.  Here,  the  SUB  curve  identifies  the  performance  of  one-shot 
ML  detection  in  complete  absence  of  multiuser  interference  (K  =  1  user  signal).  The  study  indicates  that  the 
(4m  1)  long  Gold  set  and  the  (4m+l)-long  doubly  optimal  set  perform  quite  similarly  for  both  lightly  loaded 
(K  =  3)  and  heavily  loaded  (K  =  62,  64)  systems.  We  notice  the  rather  disappointing  performance  of  both 
spreading  code  sets  especially  when  the  system  is  heavily  loaded,  which  -from  an  optimistic  point  of  view-  can  be 
seen  as  significant  room  for  improvement  in  signature  set  design  for  asynchronous/multipath  channels.  The  exact 
same  study  is  repeated  in  Fig.  6  for  a  synchronous  link  with  three  unit-gain  paths  and  indicates  that  a  primary 
reason  for  the  performance  deterioration  exhibited  in  Fig.  5  by  both  code  sets  is  asynchronism  (a  secondary 
reason  is  the  few  multipaths  that  are  present).  We  recall  that  the  “4771-1-1”  codes  of  Ref.  8  9  11  have  not  been 
“optimized”  in  any  way  for  asynchronous/multipath  transmissions.  The  Gold  codes  are  instead  presented  in 
the  literature  as  optimized  but  under  a  cyclic-shift  cross-correlation  and  auto-correlation  measure  which  can  be 
argued  to  be  inappropriate  for  asynchronous  and  multipath,  correspondingly,  DS-CDMA  channels.1 

5.  SUMMARY  AND  CONCLUSIONS 

This  paper  presented  a  theoretical  study  of  the  code  division  multiplexing  performance  of  doubly  optimal  binary 
signature  sets  of  odd  length.  These  are  sets,  we  recall,  that  have  minimum  both  total-squared-correlation  (TSC) 
and  maximum-squared-correlation  (MSC)  at  the  same  time.8,9’ 11  We  examined  separately  the  doubly  optimal 

'Gold  spreading  codes15  possess  preferred  cyclic-shift  cross-correlation  and  autocorrelation  properties.  However  they 
do  not  guarantee  good  zero-padded-shift  cross-correlation  and  autocorrelation  properties.  The  latter  measure  may 
represent  better  the  needs  of  DS-CDMA  systems  with  asynchronous  links  over  single  or  multipath  channels. 
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Figure  6.  Same  study  as  in  Fig.  5  but  for  a  synchronous  link  (still  three  unit-gain  paths). 

sets  with  signature  length  of  the  form  4m— 1  and  4m -fl,  m  =  1,2,...,  and  we  noted  that  the  familiar  Gold  sets15 
fall  under  the  “4m- 1”  class  of  doubly  optimal  sets. 

First,  we  derived  explicit  closed-form  formulas  for  the  signature  cross-correlation  matrix  of  such  sets,  its 
eigenvalues,  and  its  inverse.  Based  on  these  formulas,  we  were  able  to  obtain  analytic  expressions  for  the  BER 
of  decorrelating  filters,  the  (maximum)  output  SINR  of  MMSE  filters,  and  the  total  asymptotic  efficiency  of 
synchronous  code  division  multiplexing  systems.  The  calculated  across-the-board  superiority  of  the  “4m +1” 
signature  sets  is  intriguing.  Combined  with  the  indicated  similar  to  Gold  sets  performance  under  asynchronous 
transmissions,  this  strong  proven  superiority  for  (near-) synchronous  links  of  the  “4ra+l”  sets  questions  seriously 
the  status  quo  and  the  popularity  of  Gold  sets  (or  any  other  “4m— 1”  doubly  optimal  candidate  set  to  that 
respect).  Regretfully,  neither  the  “4m+l”  codes  nor  the  Gold  codes  have  what  we  might  call  satisfactory 
performance  for  heavy- loaded  asynchronous  transmissions.  The  “4m+l”  doubly  optimal  codes  have  not  been 
optimized  in  any  way  for  asynchronous  transmissions.  The  Gold  codes  are  presented  as  optimized  but  under  a 
cyclic-shift  cross-correlation  metric  that,  apparently,  may  be  criticized  as  inappropriate. 
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Abstract — The  prohibitive  -exponential  in  the  number  of  users- 
computational  complexity  of  the  maximum  likelihood  (ML)  mul¬ 
tiuser  detector  for  direct-sequence  code-division-multiple-access 
(DS/CDMA)  communications  has  fueled  an  extensive  research 
effort  for  the  development  of  low  complexity  multiuser  detection 
alternatives.  In  this  article,  we  show  that  we  can  efficiently  and 
effectively  approach  the  error  rate  performance  of  the  optimum 
multiuser  detector  as  follows.  We  utilize  a  multiuser  zero-forcing 
or  minimum-mean-square-error  (MMSE)  linear  filter  as  a  pre¬ 
processor  and  we  establish  that  the  output  magnitudes,  when 
properly  scaled,  provide  a  reliability  measure  for  each  user  bit  de¬ 
cision.  Then,  we  prepare  an  ordered  reliability-based  error  search 
sequence  of  length  linear  in  the  number  of  users  that  returns  the 
most  likely  user  bit  vector  among  all  visited  options.  Numerical 
and  simulation  studies  for  moderately  loaded  systems  that  permit 
exact  implementation  of  the  optimum  detector  indicate  that  the 
error  rate  performance  of  the  optimum  and  the  proposed  detector 
are  nearly  indistinguishable  over  the  whole  pre-detection  signal- 
to-noise  ratio  (SNR)  range  of  practical  interest  Similar  studies  for 
higher  user  loads  (that  prohibit  comparisons  with  the  optimum 
detector)  demonstrate  error  rate  performance  gains  of  orders  of 
magnitude  in  comparison  with  straight  decorrelating  or  MMSE 
multiuser  detection. 

Index  Terms — Code-division-multiaccess,  decorrelation,  maxi¬ 
mum  likelihood  detection,  mean  square  error  methods,  multiuser 
channels,  reliability,  soft  decision. 

I.  Introduction 

IN  direct-sequence  code-division-multiple-access 
(DS/CDMA)  systems,  multiple  users  transmit  information 
bit  sequences  over  a  common  channel  utilizing  distinct 
individually  assigned  signature  waveforms.  In  the  presence 
of  strong  interferes  and/or  high  cross-correlation  signature 
waveforms,  the  performance  of  the  conventional  single-user 
signature  matched-filter  detector  degrades  significantly.  On 
the  other  hand,  if  the  user  signatures,  energies,  and  the 
channel  noise  power  are  known  at  the  receiver  end,  use 
of  the  maximum-likelihood  (ML)  multiuser  detector  [1] 
guarantees  minimum  probability  of  error  system  performance. 
ML  optimum  multiuser  detection,  however,  comes  with 
computational  complexity  that  grows  exponentially  with  the 
number  of  uses. 

In  search  of  a  satisfactory  trade-off  between  performance 
and  complexity,  numerous  sub-optimum  multiuser  detectors 
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have  been  proposed.  Two  linear  multiuser  detectors  that  are 
of  particular  interest  to  our  work  are  the  familiar  zero-forcing 
decorrelating  detector  [2]  and  the  minimum-mean-square-error 
(MMSE)  detector  [3]-[6].  Both  are  known  to  be  near-far  resis¬ 
tant  [7].  The  decorrelator  is  the  maximum  likelihood  estimator 
of  the  energy  modulated  user  bits  for  additive  white  Gaussian 
noise  (AWGN)  multiple-access  channels.  The  MMSE  receiver 
is  the  linear  filter  solution  that  maximizes  the  pre-detection 
signal-to-interference-plus-noise  ratio  (SINR)  of  each  user. 

Recently,  there  has  been  an  effort  to  directly  approximate 
the  ML  multiuser  detector  in  an  efficient  and  effective  manner. 
Examples  of  such  work  are  the  “greedy”  detector  in  [8] 
which  partitions  the  likelihood  metric  function  and  proceeds 
with  “part-by-part”  maximization  and  the  neighbor  searching 
mechanisms  in  [9],  [10],  The  “MK-face”  detector  in  [9] 
searches  within  a  pre-calculated  set  of  neighboring  candidates. 
The  gradient  search  algorithm  in  [10]  happens  to  coincide  with 
the  “K-face”  detector  of  [9], 

As  a  combinatorial  optimization  problem,  ML  multiuser 
detection  has  certain  similarities  with  ML  decoding  of  bi¬ 
nary  linear  block  codes.  One  of  the  earliest,  well  known 
low-complexity  algorithms  for  sub-ML  block-code  decoding 
is  Chase’s  algorithm  [11].  The  Chase  procedure  selects  a 
small  fixed  number  of  bit  decision  perturbations  based  on 
reliability  measurements  which  are  supposed  to  be  part  of  the 
demodulator  output  (usually  directly  related  to  the  sufficient 
decision  statistic).  In  this  paper,  we  follow  an  analogous 
approach  to  tackle  the  ML  multiuser  detection  problem.  First, 
we  use  a  linear  multiuser  detector  (such  as  the  decorrelator 
or  MMSE  detector)  to  obtain  an  initial  multiuser  bit  decision. 
The  soft  output  vector,  before  being  discarded,  is  processed 
to  derive  the  corresponding  reliability  measurement  vector. 
Next,  as  dictated  by  the  reliability  vector,  a  sequence  of  “error 
patterns”  is  generated  to  “correct”  the  initial  decision  vector 
and  produce  multiuser  bit  combination  candidates  that  include 
the  maximum  likelihood  solution  with  high  probability.  Final 
decision  is  made  by  simply  choosing  the  bit  combination  with 
the  highest  likelihood  among  all  candidates.  When  the  length 
of  the  error  pattern  sequence  is  in  the  order  of  the  number  of 
users,  the  proposed  algorithm  exhibits  near-ML  performance 
at  remarkably  reduced  computational  complexity  compared  to 
the  ML-optimum  multiuser  detector.1 

We  organize  the  rest  of  the  paper  as  follows.  In  Section  n, 
we  present  our  signal  model  and  notation.  The  near-ML 

Mother  coding  theory  inspired  multiuser  detector  was  presented  recently 
in  [12].  In  contrast  to  the  ML  approximations  in  [8]-[10]  and  the  work  herein, 
the  multiuser  detector  of  [12]  is  a  complete  exponential-complexity  depth-first 
tree  search  implementation  of  the  full  ML  decision  rule  based  on  the  lattice 
decoder  developed  in  [13]. 
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multiuser  detection  algorithm  is  developed  in  Section  III. 
Simulation  studies  are  presented  in  Section  IV  and  a  few  final 
conclusions  are  drawn  in  Section  V. 

II.  Signal  Model  and  Notation 

For  the  sake  of  clarity  in  presentation,  we  consider  a  syn¬ 
chronous  DS/CDMA  system  where  K  users  share  a  multiple- 
access  AWGN  channel  with  power  spectral  density  a2.  Our 
developments  can  be  generalized  to  cover  near-ML  block 
multiuser  detection  for  asynchronous  systems  by  creating  error 
pattern  sequences  of  length  proportional  to  the  total  number 
of  bits  in  a  block.  The  general  block  diagram  representation 
of  the  receiver  that  we  develop  in  this  work  is  given  in  Fig.  1. 
The  received  signal  is  processed  by  a  front  end  that  consists 
of  a  block  of  K  signature-matched  filters.  We  denote  the  ith 
bit  period  output  of  the  matched  filter  bank  by  y(i)  =  (f), 

•  •  •  tlrir(t)]  (T  denotes  the  transpose  operation)  and 
we  write 

y(i)  =  RE(i)b(i)  +  n(i),  t  =  1,  2,  . . .  ,  (1) 

where  b(t)  =  MO.fcW.-.MOf  €  {±1}* 

is  the  tth_  period  information  bit  vector,  E(i)  = 
diagQv^W.v^W.-'-.V^)])  is  th t  K  x  K 
user  signal  amplitude  matrix,  R  is  the  Kx  K  signature  cross- 
correlation  matrix  and  n(i)  =  [ni(i),n2(i), . .  •  ,nK{i)f  is 
the  filtered  noise  vector.  The  ML  optimum  multiuser  detector 
[1]  is 

y)  =  2brEy  -  bTEREb)  (2) 

where  for  notational  simplicity  we  dropped  the  index  *  with 
the  understanding  that  the  decision  rule  in  (2)  refers  to  a  given 
information  bit  interval. 


III.  Near-ML  Multiuser  Detection 

We  switch  now  our  attention  to  the  linear  operator  L  and  the 
“Search  &  Decision”  blocks  in  Fig.  1.  The  linear  operator  can 
be  viewed  as  a  multiuser  detector  that  operates  on  the  output 
of  the  matched-filter  bank  y  to  produce  an  initial  decision  b<°> 
for  the  transmitted  bit  vector  b.  Alongside  the  initial  decision, 
we  also  extract  a  “reliability”  measurement  vector  a.  During 
the  final  Search  &  Decision”  stage,  an  ordered  sequence  of 
D  distinct  error  patterns  e*1*,  e(2),  ...,  e  {0,1}*  is 
generated  from  the  reliability  vector  oc  where  the  positive 
integer  D  is  a  system  parameter  to  be  adjusted  according  to  our 
performance  versus  complexity  requirements.  Corresponding 
to  each  error  pattern  e«,  d  =  1,  2,  ....  D,  a  new  tentative 
decision  b  is  produced.  The  final  decision  output  becomes 


htarw.  =  arg  max  fi(b,  y) 
be{bTO,b<*> . bW} 


(3) 


As  we  will  see  later  in  Section  IV,  when  L  is  the  decorrelating 
or  MMSE  operator,  a  small  D  (for  instance  of  the  same  order 
as  the  number  of  users/bits  to  be  detected)  results  in  bit-error- 
rate  system  performance  that  is  nearly  indistinguishable  from 
the  maximum  likelihood  detector  in  operational  environments 
of  practical  interest. 


A.  Initial  Decision  and  Reliability  Measurement 

We  use  a  linear  operator  (multiuser  detector)  LKxK  to 
obtain  the  initial  bit  decision  vector  b<0)  =  sgn(Ly).  To 
exploit  the  reliability  information  contained  in  the  linear 
operator/filter  output,  we  define  for  each  initial  bit  decision 
b)  ;  the  corresponding  soft  output 

i  =  i,2  (4) 

where  lj  is  the  jth  row  of  the  matrix  L.  For  convenience,  we 
also  define  w J  =  ijRE  and  substitute  (1)  in  (4)  to  calculate 


n 

di  =  =  »ah+  £  WjA+lJn,  j  =  1,  2, 


K. 


(5) 


The  first  term  on  the  right-hand  side  of  (5)  is  the  scaled 
ith  transmitted  bit  of  interest.  The  last  term,  lTn  =  n  ,  is 
zero-mean  filtered  Gaussian  noise  with  variance  E{n2}  = 

er2lj Rlj  =  <r2r  The  middle  term  Wjibi  =  Zj 

contains  the  information  bits  transmitted  by  the  other  users 
(multiple-access-interference  or  MAI)  and  is  a  zero-mean 
random  variable  with  variance  E{z*\  =  -*#,2  _ 

..o  «  A  ~  wji  ~ 


w, 


“  -  < 


zj  •  We  define  the  reliability  value  aj  of 

the  initial  bit  decision  bf]  as  the  log-likelihood  ratio  a,  = 

.  _  1 

2  /(d>|i/=-S<0)}  W*iere  /  (“il&j)  denotes  the  probability 

density  function  of  dj  conditioned  on  bj.  If  the  filtered  MAI 
variance  E{zj}  is  relatively  small,  the  soft  output  d,  is  well 
approximated  [14],  [15]  by  a  Gaussian  random  variable  with 
mean  E  {dj }  =  Wjjbj  and  variance  E  {dj2}  =  a2  +  a2 
Then,  the  reliability  value  aj  simplifies  to  a  scaled  version  of 


2  /(tfi  I* —«?*>) 


ai+< 


_  wn 


l“il 


The  generated  reliability  vector  a  =  [alt  •  •  •  ,a/f]TSs 
passed  to  the  “Search  &  Decision”  stage  along  with’ the  hard- 
limited  initial  “guess”  b(°). 


When  L  is  the  decorrelating  or  MMSE  filter  operator,  we 
expect  intuitively  that  a  small  perturbation  of  the  initial  vector 
,  may  Produce  the  maximum  likelihood  decision  b^.  With 
this  motivation,  we  generate  a  sequence  of  error  patterns 

e  (O'1)  •  <>-  1.  2 .  Wh.re  .»  =  1  stMds  f0[ 

an  error  in  bit  position  j.  We  apply  the  error  sequence  e<d> 
on  the  initial  bit  vector  b<°>  to  create  the  bit  vector  sequence 
bw  as  follows: 


if  =Sf©ef, 


i  =  1,  2,  ...,  K, 


(7) 


where  ©  denotes  the  error  correction  operation  b  ©  e  = 
^  ’  (~ l)e»  b  €  {±1},  e  €  {0, 1}.  Naturally,  the  error  sequence 
generation  criterion  will  be  the  likelihood  of  b  =  b(°>©e  with 
observation  the  soft-output  vector  a  =  Ly,  /  (d|b  =  b).  If, 


3 


approximately,  we  treat  the  soft  outputs  dj  as  conditionally  in¬ 
dependent  random  variables,  the  likelihood  function  becomes 
IljLi  /  (®j\bj  =  h'j  and  the  log-likelihood  ratio  is 


A(b) 


1^,  f  =  h) 


5Z  ai  -  L  ai 


^  t(-n>  i\  ^  3  ^  : 

i=l  /  {<Xj\bj  =  -bj)  j:Sj=S<0)  i:S3#S<°) 


—  -  2  aJ- 

j=l  j:ci=l 


(8) 


We  immediately  observe  that  the  first  term  in  (8),  Y%=  i  Q;> 

is  independent  of  the  choice  of  the  error  pattern  e.  Therefore, 

the  error  pattern  sequence  should  be  designed  in  ascending 

order  of  the  other  term  <f>(e,a)  =  2J:e3=iaj-  F°r  our 

purposes  of  near-ML  multiuser  detection,  if  all  2K  error 
2<k 

patterns  {e^}^  are  in  ascending  order  under  the  0(-,a) 
key 


< f>  (e^,  a)  <  <t>  (e<d+1>,  a)  ,  d  =  1,  2,  . . . ,  2*  - 1,  (9) 

we  will  consider  and  process  only  the  first  D  elements  of  the 
sequence. 

We  now  derive  an  algorithm  that  produces  the  first  D  0(-,  •)- 
ordered  error  patterns  with  computational  complexity  linear  in 
the  number  of  bits,  O(DK).  The  origins  of  our  algorithm  are 
in  the  “subset-sum  problem”  where  the  objective  is  to  find 
all  subsets  of  a  set  of  numbers  that  have  sum  of  elements 
less  than  or  equal  to  a  given  number  [16],  [17].  Suppose  that 
the  reliability  values  a^,  j  =  1,  2,  K,  are  such  that 
othx  <  othi  <  •  *  *  <  olhk  •  Define  the  error  pattern  sequences 
£p  =  {e(P‘>},  p  =  0,  1,  2,  K,  t  =  1,  2,  2 p, 

where  {e^p‘^ }  is  ordered  under  the  key  <j>(-,a) 

<^(e(pt),a)  <  <A(e(,,‘+,),a)  ,  t  =  1,  2,  ....  2P  -  1, 
v  7  v  (10) 
and  satisfies 


=  0  for  every  h  e  {/ip+i,  •  •  • ,  h k}  and  1  <  t  <  2P. 
n  (11) 
Here,  is  the  hth  bit  in  the  error  pattern  and  (1 1)  states 

that  may  include  errors  only  in  the  p  lowest  reliability 

positions  of  the  initial  bit  decision  vector  b^°\  Define  also  the 
error  pattern  sequences  £p  =  {e'^l },  p  =  0,  1,  2,  . . . ,  K  - 
1,  t  =  1,  2,  . . . ,  2P,  where  |e'(pt)}  is  such  that 


/  (pt)  _  f  !■>  if  h  —  hp+ 1 

h  \  e£Pt\  otherwise. 


(12) 


Therefore,  each  e'^  contains  exactly  one  more  error  in  bit 
position  hp+i  than  the  corresponding  error  pattern  e^Pt\  It 
follows  that  (f>  (e'(pt\a)  =  <f>  (e(pt\a)  +a/lp+1  and  {e'(pt)} 
is  also  ordered  under  the  key  0(*,a). 

At  this  point,  it  is  important  to  observe  the  following.  If  we 
merge  the  two  ordered  sequences  £p  =  {e^}  and  £p  = 
in  increasing  order  of  the  same  key  0(*,a),  then  we  will 
obtain  £p+i.  Therefore,  iterative  generation  of  £fp  from  £p  and 
merging  to  form  £p+i  for  p  =  0,  1,  . . . ,  K  —  l  leads  to  £k 
where  £k  is  the  whole  set  of  2K  error  patterns  ordered  under 


the  key  0(-,  a).  Since  only  the  first  D  error  patterns  in  £k  are 
needed,  we  can  simplify  this  process  greatly.  We  observe  that 
the  first  D  elements  in  £k  come  from  the  first  D  elements  of 
£k- i  and  the  first  D  elements2  of  £fK-V  Therefore,  we  can 
truncate  the  length  of  £k-\  to  D  (which  also  limits  the  length 
of  £*k-\  to  D).  By  induction,  we  conclude  that  every  sequence 
£q>  £\> ..  .*  £k  can  be  shortened  to  length  D .  Considering  the 
involved  computational  complexity,  we  need  up  to  D  additions 
to  calculate  </>(■,  a)  and  another  D  comparisons  to  merge  £p 
and  £p  in  each  step.  Therefore,  the  computational  complexity 
is  O(DK). 

In  summary,  the  proposed  multiuser  detection  system  in 
Fig.  1  operates  as  follows: 

1)  Apply  the  linear  operator/filter  L  to  the  matched-filter 
bank  output  y  to  obtain  the  initial  decision  b(°^  and  the 
reliability  measures  {aj}  in  (6). 

2)  Sort  {aj}  in  increasing  order. 

3)  Initialize  p  <-  0,  £o  =  (e(1)  =  [0  0  0]}, 

0  (eW,o)  =  0. 

4)  Create  £p  =  <  e,(*^=eW+[£— J 0  1  0j^0  ] :  e  £p 

V  hp+  i—l  K— /ip+i 

Calculate  <j>  (e'^,  a)  =  <f>  (e^,  a)  -f  cthP+i  • 

5)  Merge  £p  and  £p  to  form  £p+\  in  ascending  order  under 
key  0(*,a).  Truncate  the  length  of  £p+i  to  D  if  longer 
than  D . 

6)  If  p  <  K  -  1,  then  set  p  <-  p  +  1  and  go  back  to  Step 
4.  Else  go  to  Step  7. 

7)  For  d  =  1,  2,  . . . ,  D  apply  to  the  initial  decision 

as  in  (7).  Produce  the  final  decision  Iwml  by  (3). 

The  overall  system  computational  complexity  can  be  cal¬ 
culated  as  follows.  The  complexity  of  applying  the  linear 
operator  L  to  the  matched-filter  output  y  is  O  (K2).  Sorting 
the  reliability  values  costs  O  (Klog2  K).  As  stated  earlier,  the 
complexity  of  generating  the  ordered  sequence  {e(d)}^=1  is 
0(DK ).  Finally,  f2(*,  •)  is  calculated  D  times  with  complex¬ 
ity,  again,  0(DK).  Therefore,  the  complexity  of  the  whole 
scheme  is  approximately  o(jC(K  +  2D  +  log2  Hf))  per  bit 
period.  The  cost  of  the  “Near-ML  Search  &  Decide^block  in 
Fig.  1  is  barely  more  than  twice  the  cost  of  the  Ly  operation  if 
D  is  chosen  equal  to  K  (that  is,  if  D  is  equal  to  the  number  of 
user  bits  to  be  detected).  As  we  will  see  later  in  Section  IV, 
when  the  linear  operator  L  is  chosen  appropriately  a  value 
of  D  equal  to  K  or  -more  general-  a  few  times  K  can  be 
sufficient  for  the  algorithm  to  maintain  near-ML  performance 
for  all  practical  purposes. 

C.  Selection  of  the  Linear  Operator 

So  far,  we  have  intentionally  treated  the  linear  operator  L  as 
arbitrary.  Yet,  it  is  important  to  choose  an  operator/filter  that 
produces  initial  decisions  with  low  probability  of  error  such 
that  only  a  small  perturbation  is  needed  to  approach  the  ML 

2In  fact,  only  the  first  [^J  elements  of  Z'K _x  can  contribute  to  the  first  D 
elements  of  £k  but  this  does  not  affect  our  conclusion  on  the  rough  order  of 
the  involved  computational  complexity  and  may  complicate  our  presentation 
unnecessarily. 


optimum  decision  vector.  Conversely,  if  the  initial  decisions 
are  poor,  then  to  achieve  the  same  bit-error-rate  performance 
level  we  will  need  a  larger  search  length  parameter  D  and  that 
will  thereby  increase  the  computational  complexity. 

Besides  the  initial  performance  considerations,  the  condition 
that  the  variance  of  the  MAI  term  Zj  =  (n 

(5)  is  relatively  small  should  also  be  fulfilled.  This  directly 
affects  the  accuracy  of  the  Gaussian  approximation  of  the  soft 
output  dj  which  in  turn  affects  the  quality  of  the  reliability 
measure  <*_,  in  (6).  There  are  two  well-researched  linear  filters 
that  can  satisfy  the  quality  of  initial  decision  and  small  MAI 
variance  requirements:  the  decorrelating  and  MMSE  filters. 
The  decorrelating  filter  =  R-1,  leading  to  zero  errors 
in  the  absence  of  noise,  has  perfect  near-far  resistance  and 
is  the  maximum  likelihood  solution  for  real-valued  “symbol” 
vectors.  If  we  use  Iw  =  R1  in  (4),  (5),  we  obtain 


~  Bii*  ans  =  and  <r2  =  o.  Because  MAI 

is  completely  cancelled,  the  Gaussian  approximation  is  not 
needed  anymore.  The  reliability  measures  in  (6)  take  the  form 

E  •  • 

N>  J  =  l,  2,  K,  (13) 


a,  = 


!(R-i) 


'li 


with  no  approximation.  The  MMSE  multiuser  filter  L,^  = 
^  ®  )  minimizes  the  mean  norm  square  error  be¬ 
tween  its  output  and  the  true  transmitted  information  bit  vector 
and  maximizes  the  output  SINR  per  user.  However,  the  filtered 

^ ‘e™  “  is  not  zero:  *  0.  Still, 

the  MMSE  filtered  MAI  is  approximately  Gaussian  [14], [15] 
and  the  reliability  measure  in  (6)  maintains  its  accuracy.' 

As  a  final  comment,  it  is  clear  that  technically  the  soft 
outputs  dj,  j  =  1,  2,  K,  are  not  independent  and 
their  degree  of  dependency  can  affect  the  quality  of  the 
log-likelihood  ratio  approximation  in  (8)  and,  in  turn,  the 
quality  of  the  generated  sequence  of  error  patterns.  When  the 
linear  operator  L  is  the  decorrelator,  it  is  easy  to  calculate 
the  correlation  coefficient  ptj  between  and  d„  i  4  4; 

j  ~  where 

SNR<  =  p.  In  most  cases  of  practical  interest,  |p0  |  is 
less  than  10  2  and  the  independence  approximation  is  well 
justified.  In  such  cases,  we  found  (see  for  example  Section 
IV)  that  D  =  K  is  sufficient  to  approximate  very  closely  the 
performance  of  the  true  ML  detector.  In  extreme  situations, 
such  as  pre-detection  SNR  values  less  than  OdB  and  high 
signature  cross-correlations  (greater  than  0.5),  | Pij\  can  be 
greater  than  0.1  and  thus  non-negligible.  In  these  cases,  a 
greater  D  should  be  used  to  approach  ML  performance.  For 
MMSE  preprocessors,  we  can  calculate  \pij\  numerically  and, 
again,  adjust  D  accordingly. 


filter  as  the  pre-processor  in  our  proposed  system  and  we 
fix  the  length  of  the  error  pattern  sequence  to  D  =  K  = 
15  (hence,  the  computational  complexity  per  bit  interval  is 
0(DK)  =  0(K2)).  For  purposes  of  comparison,  we  include 
(i)  the  ML  optimum  multiuser  detector  [1]  (which  serves  as 
a  BER  lower  bound),  (ii)  the  stand-alone  multiuser  MMSE 
detector  [3]-[6],  (Hi)  the  “greedy”  multiuser  detector  in  [8] 
with  different  memory  parameter  settings  L  =  1,2,3  and 
computational  complexity  per  bit  interval  0(LK2),  (iv)  the 
“MK-face”  detector  [9]  with  parameter  setting  MK  =  215  = 
30,  computational  complexity  per  bit  interval  0(MK2)  and 
space  complexity  0{2kMK),  and  (v)  the  “gradient  search” 
detector  [10]  that  coincides  with  the  “K-face”  algorithm  of 
[9]  and  has  computational  complexity  0(K2)  per  iteration. 
We  observe  that  in  the  relatively  low  SNR  region  (4  -  12dB) 
the  greedy,  the  MK-face,  and  the  proposed  detector  have 
performance  very  close  to  the  optimum.  We  recall  that  the 
greedy  algorithm  decides  the  value  of  the  user  information  bits 
sequentially  and  the  order  favors  high  energy  bits/users.  As  the 
SNR  of  the  “user  of  interest”  increases  relative  to  the  other 
users,  bit  decisions  are  made  early  with  the  adverse  effect  of 
an  increase  in  BER  as  seen  in  Fig.  2.  For  the  MK-face  detector, 
the  pre-calculated  neighboring  candidates  fail  to  perturb  high 
SNR  user  bits  and  the  performance  curve  degrades  to  the  pre¬ 
detector  (MMSE  in  this  case).  The  proposed  system  maintains 
“near-ML”  performance  throughout  the  range  of  this  study.  In 
Fig.  3,  we  repeat  the  study  of  Fig.  2  and  we  examine  the 
BER  of  the  proposed  detector  as  a  function  of  the  length  of 
the  error  pattern  sequence  D  at  SNRX  =  l2dB  and  14dB. 
The  proposed  algorithm  approaches  effectively  the  ML  bound 
for  modest  values  of  D  (the  number  of  bits  to  be  detected 
is  K  —  15;  D  =  1  corresponds  to  the  stand-alone  MMSE 
detector  and  D  =  215  corresponds  to  the  true  ML  detector). 

Finally,  in  Fig.  4  we  increase  the  number  of  active  users 
from  AT  =  15  to  K  =  25.  The  SNRs  of  the  24  interferes 
are  distributed  over  the  8.5  -  20dB  range  with  a  0.5dB 
increment.  The  performance  gain  of  the  proposed  “near-ML” 
algorithm  with  D  =  4K  =  100  over  the  MMSE  detector  is 
remarkable.  True  ML  detection  has  complexity  proportional  to 
2  and  we  cannot  afford  this  computational  cost  at  present 
For  this  reason,  we  replace  the  true  ML  bound  that  appears 
in  the  previous  Figs.  2  and  3  by  the  single-user  bound. 
We  also  cannot  afford  execution  of  the  “MK-face”  algorithm 
that  requires  exponential  (22&MK)  storage  space.  Instead,  we 
present  in  Fig.  4  the  performance  curves  of  the  “greedy” 
(L  =  4)  and  “gradient  search”  (“K-face”)  algorithms  [8]-[10], 

V.  Conclusions 


IV.  Simulation  Studies 

We  consider  the  DS/CDMA  signal  model  of  Section  II  with 
spreading  gain  31  and  K  =  15  users  with  Gold  signature  code 
assignments.  In  Fig.  2,  we  plot  the  bit-error-rate  (BER)  of  the 
user  of  interest  (user  1)  under  different  multiuser  detection 
schemes  as  a  function  of  its  SNR  over  the  4  -  16dB  range. 
The  SNRs  of  the  14  interferers  are  distributed  in  the  8.5-15dB 
range  with  a  0.5dB  increment  We  use  the  multiuser  MMSE 


We  described  a  new  multiuser  detection  algorithm.  A  decor¬ 
relating  or  MMSE  multiuser  filter  is  used  as  a  pre-processor 
that  provides  initial  decisions  and  reliability  measurements 
based  on  which  an  ordered  error  pattern  sequence  of  variable 
length  is  formed.  The  error  pattern  sequence  is  followed  to  its 


in ittus  study  we  choose  a  Gold  code  assignment  that  makes  the  normalized 
signature  cross-correlation  between  user  1  and  the  other  users  equal  to  -L  or 

th/taL1 detector  remarkabIe  Performance  difference  between  the  MMSEand 
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end  and  the  most  likely  bit  vector  among  all  visited  options 
is  returned.  When  the  length  of  the  pattern  sequence  is  of 
the  order  of  the  number  of  bits  to  be  detected,  the  additional 
imposed  computational  cost  compared  to  straight  decorrelating 
or  MMSE  detection  is  rather  insignificant.  Still,  in  extensive 
simulation  studies  for  both  synchronous  and  asynchronous 
links  (not  reported  herein)  we  saw  that  the  proposed  multiuser 
detection  algorithm  is  able  to  maintain  near-ML  bit-error-rate 
performance  over  the  whole  studied  SNR  range  of  interest. 
There  is  strong  resemblance  between  this  scheme  and  “effi¬ 
cient**  decoding  algorithms  for  binary  linear  block  codes  [11], 
[17]. 
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Fig.  1.  The  proposed  “near-ML”  multiuser  detection  architecture. 


Fig.  2.  Bit-error-rate  versus  SNR  for  various  multiuser  detectors  (K  -  15, 
SNRi  in  8.5  to  15dB  range,  i  —  2, 3,  •  •  • ,  15). 


Fig.  3.  Bit-error-rate  versus  length  of  the  error  pattern  sequence  D  for  the 
system  of  Fig.  2  (detection  of  K  =  15  user  bits). 


Fig.  4.  Bit-error-rate  versus  SNR  performance  in  the  presence  of  severe  MAI 
(K  =  25,  SNRi  in  8.5  to  20dB  range,  i  =  2, 3,  •  •  • ,  25). 


